Apache Solr Release Notes
This file lists Solr's raw release notes with details of every change to Solr.
Most people will find the solr-upgrade-notes.adoc file more approachable.
https://github.com/apache/solr/blob/main/solr/solr-ref-guide/modules/upgrade-notes/pages/solr-upgrade-notes.adoc
- New Features (8)
- SOLR-13350, SOLR-17298: Opt-in multithreaded search execution
(Ishan Chattopadhyaya, Mark Miller, Christine Poerschke, David Smiley, noble, Gus Heck)
- SOLR-17192: Put an UpdateRequestProcessor-enforced soft-limit on the number of fields allowed in a core. The `NumFieldLimitingUpdateRequestProcessorFactory`
limit may be adjusted by raising the factory's `maxFields` setting, toggled in and out of "warning-only" mode using the `warnOnly` setting, or disabled entirely
by removing it solrconfig.xml. The limit is set at 1000 fields in the "_default" configset, but left in warning-only mode.
(David Smiley, Eric Pugh,
Jason Gerlowski)
- SOLR-17277: Circuit breakers may now be configured in a "soft" or "warnOnly" mode in order to more easily test out new thresholds. Soft breakers will log out a
message on each relevant request when tripped, but will not otherwise impact or short circuit the requests.
(Jason Gerlowski)
- SOLR-17335: New "vectorSimilarity" QParser for matching documents mased on a minimum vector similarity threshold.
(hossman)
- SOLR-10654: Introduce output of Prometheus metrics directly from Solr.
(Matthew Biscocho via David Smiley)
- SOLR-17195: Configsets now include a `minPrefixQueryTermLength` setting, which instructs Solr to reject prefix queries whose prefixes are "too short". This can
be used as one line of defense against "runaway wildcard queries" consuming too many resources. The setting is disabled ('-1') in the default configset but can be
overridden with a property ('solr.query.minPrefixLength'). Users may also override their collection-wide setting for individual queries by providing a
`minPrefixQueryTermLength` local-param.
(Jason Gerlowski, David Smiley)
- SOLR-10255: Add support for docValues to solr.BinaryField.
(Alexey Serba via Mikhail Khludnev, David Smiley)
- SOLR-17276: Prometheus Exporter: now scrapes metrics at a fixed rate instead of delay.
(Rafał Harabień)
- Improvements (16)
- SOLR-10808, SOLR-12963: The Solr schema version has been increased to 1.7.
Starting in schema version 1.7, most fields/fieldTypes that support docValues will have them enabled by default.
These field types include primitive (Numeric, Date, Bool, String, Enum, UUID), sorting (SortableTextField, SortableBinaryField, CollationField, ICUCollationField) and LatLonPointSpacialField.
This behavior can be reverted by setting the 'docValues' parameter for a field or a field type to false, the default for schema versions 1.6 and below.
Also in schema version 1.7, all fields/fieldTypes will be unable to be uninverted by default.
This behavior can be reverted by setting the 'uninvertible' parameter for a field or a field type to true, the default for schema versions 1.6 and below.
(Houston Putman, hossman)
- SOLR-17137: Enable Prometheus exporter to communicate with SSL protected Solr.
(Eivind Bergstøl via Eric Pugh)
- SOLR-16921: use -solrUrl to derive the zk host connection for bin/solr zk subcommands
(Eric Pugh)
- SOLR-14115: Add linkconfig, cluster, and updateacls as commands to SolrCLI. Allow bin/solr commands to have parity with zkcli.sh commands.
(Eric Pugh)
- SOLR-17274: Allow JSON atomic updates to use multiple modifiers or a modifier like 'set' as a field name
if child docs are not enabled.
(Calvin Smith, David Smiley)
- SOLR-17300: Http2SolrClient.Builder.withHttpClient now copies HttpListenerFactory (e.g. for auth, metrics, traces, etc.)
(Sanjay Dutt, David Smiley)
- SOLR-17109: Give security manager explicit read access to sharedLib
(Tomás Fernández Löbbe via Eric Pugh)
- SOLR-17331: OrderedNodePlacementPlugin will give an even more optimal replica placements during ReplicaMigration commands
(Houston Putman, Yohann Callea)
- SOLR-15591: Make using debugger in Solr easier by avoiding NPE in ExternalPaths.determineSourceHome.
(@charlygrappa via Eric Pugh)
- SOLR-16824: Adopt Linux standard pattern of -- for long option commands, and make all commands "kebab" formatting. I.e -zkHost is now -zk-host. The old parameters
such as -zkHost continue to be supported in the 9.x line of Solr. -u is now used to specify user credentials everywhere, this only impacts the bin/solr assert
commands "same user" check which has -u as the short form of --same-user.
(Eric Pugh, janhoy, Jason Gerlowski, Christos Malliaridis)
- SOLR-17346: Synchronise stopwords from snowball with those in Lucene
(Alastair Porter via Houston Putman)
- SOLR-16198: Introduce tabbed sections again in the Ref Guide.
(Christos Malliaridis via Eric Pugh)
- SOLR-17160: Core Admin "async" request status tracking is no longer capped at 100; it's 10k.
Statuses are now removed 5 minutes after the read of a completed/failed status. Helps collection
async backup/restore and other operations scale to 100+ shards.
(Pierre Salagnac, David Smiley)
- SOLR-10808 : The Solr schema version has been increased to 1.7. Since schema version 1.7, all fields/fieldTypes that
support docValues will have them enabled by default. This behavior can be reverted by setting
'docValues' parameter for a field or a field type to false, the default for schema versions 1.6 and below.
(Houston Putman)
- SOLR-17418: Streamline ConfigSet modification logic.
(Houston Putman, Liu Huajin)
- SOLR-17434: Avoid exposing Solr's IP in the Location header.
(David Smiley)
- Optimizations (7)
- SOLR-17257: Both Minimize Cores and the Affinity replica placement strategies would over-gather
cluster metrics. AttributeFetcherImpl was improved accordingly.
(Yohann Callea)
- SOLR-17099: Do not return spurious tags when fetching metrics from a Solr node to another.
(Pierre Salagnac)
- SOLR-17269, SOLR-17386: Prevent the "Coordinator node" feature from registering synthetic cores in ZooKeeper
(ellaeln, Patson Luk, David Smiley, Christine Poerschke)
- SOLR-17330: When not set, loadOnStartup defaults to true, which is the default choice for a core.
(Pierre Salagnac via Eric Pugh)
- SOLR-16677: Update Solr to use new Lucene 9.5 storedFields() API. This removes the use of ThreadLocal for
stored field state, reducing heap usage especially for high-core-count, high-field-count, high-thread-count
cases
(Vinayak Hegde, Christine Poerschke, Kevin Risden, David Smiley, Michael Gibney)
- SOLR-17349: SolrDocumentFetcher should always skip lazy field loading overhead if documentCache==null
(Michael Gibney)
- SOLR-17340: Add cache on top of system metrics BeanInfos to make calls to /admin/info/system faster
(Pierre Salagnac)
- Bug Fixes (20)
- SOLR-12813: subqueries should respect basic auth.
(Rudy Seitz via Eric Pugh)
- PR-2475: Fixed node listing bug in Admin UI when different hostnames start with the same front part.
(@hgdharold via Eric Pugh)
- SOLR-16659: Properly construct V2 base urls instead of replacing substring "/solr" with "/api"
(Andrey Bozhko via Eric Pugh)
- SOLR-17247: 'WWW-Authenticate' headers missing in MultiAuthPlugin
(Lamine Idjeraoui via Eric Pugh)
- SOLR-17304: Respect PKG_VERSIONS constraints in params.json when loading the schema plugins
(Andrey Bozhko)
- SOLR-17118: Simplify CoreContainerProvider initialization and JettySolrRunner. Avoid deadlock/hang.
(David Smiley)
- SOLR-17304: PKG_VERSIONS not honored when loading the schema plugins
(#2471)
- SOLR-17317: v2 APIs no longer return the incorrect content-type when 'wt=json' specified
(Jason Gerlowski)
- SOLR-17315: Fixed "next step" example urls generated by bin/solr create commands to use the newly created collection/core name.
(Eric Pugh)
- SOLR-17326: Generated v2 'SolrRequest' implementations now all serialize to the correct response type
(Christos Malliaridis via Jason Gerlowski)
- SOLR-16962: Restore ability to configure tlog directory. Changes largely focus on UpdateLog and
HdfsUpdateLog. There is no anticipated change in existing behavior, other than that the `dir` parameter
to the `<updateLog>` element in `solrconfig.xml`, which has long been silently ignored, will now be
respected
(Michael Gibney, David Smiley)
- SOLR-17255: Fix bugs in SolrParams.toLocalParamsString()
(hossman)
- SOLR-17118: Simplify CoreContainerProvider initialization and JettySolrRunner. Avoid deadlock/hang.
(David Smiley)
- SOLR-17333: Rate-limiting feature: fix live-update of config
(Michael Gibney)
- SOLR-17367: Restore the use of -params option to PostTool.
(Bostoi via Eric Pugh)
- SOLR-17369: Fix "flags" usage in FunctionQParser that caused some issues in vectorSimilarity() with BYTE vector constants
(hossman)
- SOLR-17337: Display all custom distributed stages in debug output.
(Torsten Bøgh Köster, Christine Poerschke)
- SOLR-17394: Detect and handle non-200 HTTP status codes for requests made by IndexFetcher
(Jason Gerlowski)
- SOLR-17391: Fixed performance regression of misconfigured threadpools from SOLR-16879 (Solr 9.4).
Shard splits and concurrent/large collection backup/restore performance was serial. UpdateLog
replay was a little suboptimal in thread usage too.
(Pierre Salagnac, Hakan Özler, David Smiley)
- SOLR-17417: Remove unnecessary code in PKIAuthPlugin and HttpSolrCall
(Houston Putman, janhoy, Liu Huajin)
- Dependency Upgrades (27)
- PR#2512: Update dependency com.carrotsearch:hppc to v0.10.0
(solrbot)
- PR#2520: Update io.dropwizard.metrics:* to v4.2.26
(solrbot)
- PR#2572: Update dependency org.apache.commons:commons-lang3 to v3.15.0
(solrbot)
- PR#2434: Update dependency org.bouncycastle:bcpkix-jdk18on to v1.78.1
(solrbot)
- PR#2140: Update org.apache.calcite to 1.37.0
(solrbot)
- PR#2269: Update dependency org.apache.hadoop.thirdparty:hadoop-shaded-guava to v1.2.0
(solrbot)
- PR#2053: Update dependency com.google.cloud:google-cloud-bom to v0.224.0
(solrbot)
- PR#2056: Update software.amazon.awssdk:* to v2.26.19
(solrbot)
- PR#2131: Update dependency org.jctools:jctools-core to v4.0.5
(solrbot)
- PR#2568: Update dependency commons-codec:commons-codec to v1.17.1
(solrbot)
- PR#2521: Update dependency org.apache.commons:commons-configuration2 to v2.11.0
(solrbot)
- PR#2314: Update io.grpc:grpc-* to v1.65.1
(solrbot)
- PR#2413: Update io.netty:* to v4.1.111.Final
(solrbot)
- PR#2406: Update org.slf4j:* to v2.0.13
(solrbot)
- PR#2182: Update org.apache.curator:* to v5.7.0
(solrbot)
- PR#2468: Update org.eclipse.jetty*:* to v10.0.22
(solrbot)
- PR#2560: Update dependency com.fasterxml.jackson:jackson-bom to v2.17.2
(solrbot)
- PR#2490: Update org.mockito:mockito* to v5.12.0
(solrbot)
- PR#2373: Update dependency io.swagger.core.v3:swagger-annotations-jakarta to v2.2.22
(solrbot)
- PR#2346: Update dependency com.fasterxml.jackson:jackson-bom to v2.17.1
(solrbot)
- PR#2440: Update dependency org.semver4j:semver4j to v5.3.0
(solrbot)
- PR#2481: Update dependency commons-cli:commons-cli to v1.8.0
(solrbot)
- PR#2426: Update dependency commons-codec:commons-codec to v1.17.0
(solrbot)
- PR#2341: Update org.apache.zookeeper:* to v3.9.2
(solrbot)
- PR#2308: Update dependency org.apache.commons:commons-compress to v1.26.1
(solrbot)
- PR#2408: Update dependency commons-cli:commons-cli to v1.7.0
(solrbot)
- SOLR-17325: Upgrade Lucene to 9.11.1
(hossman)
- Other Changes (9)
- SOLR-17248: Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage
(Lamine Idjeraoui via Eric Pugh)
- SOLR-16505: Use Jetty HTTP2 for index replication and other "recovery" operations
(Sanjay Dutt, David Smiley)
- GITHUB#2454: Refactor preparePutOrPost method in HttpJdkSolrClient
(Andy Webb)
- SOLR-16503: Use Jetty HTTP2 for SyncStrategy and PeerSyncWithLeader for "recovery" operations
(Sanjay Dutt, David Smiley)
- PR#2524: QueryResult refactoring so that it's only returned from SolrIndexSearcher instead of
being provided to it. Deprecated APIs in 9x; should be removed later.
(David Smiley)
- SOLR-17347: EnvUtils: Removed "env" methods, as they were problematic. Code should almost never
refer to env vars.
(David Smiley)
- SOLR-15831: Refactor bin/solr and bin/solr.cmd commands integration with AuthTool, DeleteTool, and PackageTool to delegate arg parsing to Java code. Removed limitation of PackageTool only being executed on an active Solr node.
(Eric Pugh)
- SOLR-17322: Once again allow rank queries to use custom TopDocsCollectors that operate on types that extend
ScoreDocs (covariant generic types) broken in Solr 9.0.
(Stephen Woods via Christine Poerschke)
- SOLR-16996: Update Solr Exporter for Prometheus cli to use commons-cli instead of argparse4j.
(Christos Malliaridis via Eric Pugh)
- Bug Fixes (6)
- SOLR-17261: Remove unintended timeout of 60 seconds for core loading.
(Houston Putman)
- SOLR-17049: Actually mark all replicas down at startup and truly wait for them.
This includes replicas that might not exist anymore locally.
(Houston Putman, Vincent Primault)
- SOLR-17263: Fix 'Illegal character in query' exception in HttpJdkSolrClient
(Andy Webb)
- SOLR-17275: SolrJ ZkClientClusterStateProvider, revert SOLR-17153 for perf regression when aliases are used.
(Aparna Suresh)
- SOLR-17296: Remove (broken) singlePass attempt when reRankScale + debug is used, and fix underlying NPE.
(hossman)
- SOLR-17307: Use the system file separator instead of an explicit '/' in CachingDirectoryFactory
(Houston Putman, hossman)
- Dependency Upgrades (1)
- PR#2308: Update dependency org.apache.commons:commons-compress to v1.26.1
(solrbot)
- New Features (4)
- SOLR-17141: Implement 'cpuAllowed' query parameter to limit the maximum CPU usage by a running query.
(Andrzej Bialecki, Gus Heck, David Smiley)
- SOLR-599: Add a new SolrJ client using the JDK’s built-in Http Client.
(James Dyer)
- SOLR-16403: A new cluster singleton plugin to automatically remove inactive shards.
(Paul McArthur, David Smiley)
- SOLR-16466: Admin UI - Make it optional to sort list of commandline args
(Shawn Heisey, Vincenzo D'Amore via Christine Poerschke)
- Improvements (14)
- SOLR-17119: When registering or updating a ConfigurablePlugin through the `/cluster/plugin` API,
config validation exceptions are now propagated to the callers.
(Yohann Callea)
- SOLR-16699: Add Collection creation time to CLUSTERSTATUS and COLSTATUS API responses
(Julien Pilourdault, Paul McArthur, David Smiley)
- SOLR-17038: Expose the term count in /admin/segments handler if fieldInfo=true.
(Rahul Goswami via David Smiley and Christine Poerschke)
- SOLR-17146: Add DelegatingBackupRepository and alternative checksum verification
(Bruno Roustant)
- SOLR-16858: KnnQParser's "Pre-Filtering" behavior is now controlable via local params
(hossman)
- SOLR-17145: The INSTALLSHARDDATA API now includes a 'requestid' field when run asynchronously
(Jason Gerlowski)
- SOLR-17159: bin/solr post now has proper unit testing. Users can specify a --dry-run option to
simulate posting documents without sending them to Solr.
(Eric Pugh)
- SOLR-17058: Add 'distrib.statsCache' parameter to disable distributed stats requests at query time.
(Wei Wang, Mikhail Khludnev)
- SOLR-16138: Throw a exception when issuing a streaming expression and all cores are down instead of returning 0 documents.
(Antoine Bursaux via Eric Pugh)
- SOLR-17172: Add QueryLimits termination to the existing heavy SearchComponent-s. This allows query limits (e.g. timeAllowed,
cpuAllowed) to terminate expensive operations within components if limits are exceeded.
(Andrzej Bialecki)
- SOLR-17164: Add 2 arg variant of vectorSimilarity() function
(Sanjay Dutt, hossman)
- SOLR-17211: New SolrJ JDK client supports Async
(James Dyer)
- SOLR-17204: REPLACENODE now supports the source node not being live
(Vincent Primault)
- SOLR-14763: Add a CompletableFuture based asynchronous API to Http2SolrClient, HttpJdkSolrClient and LBHttp2SolrClient.
The previous asynchronous API is deprecated.
(Rishi Sankar, James Dyer)
- Optimizations (2)
- SOLR-17144: Close searcherExecutor thread per core after 1 minute
(Pierre Salagnac, Christine Poerschke)
- GITHUB#2217: Scale to 10K+ collections better in ZkStateReader.refreshCollectionsList
(David Smiley)
- Bug Fixes (20)
- SOLR-17153: CloudSolrClient could fail a request immediately following a collection creation. Required double-checking the collection doesn’t exist.
(Aparna Suresh via David Smiley)
- SOLR-17152: Better alignment of Admin UI graph
(janhoy)
- SOLR-17148: Fixing Config API overlay property enabling or disabling the cache
(Sanjay Dutt, hossman, Eric Pugh)
- PR#2320: Avoid NullPointerException in SolrJ client due to missing
EnvToSyspropMappings.properties file
(janhoy)
- SOLR-17186: Streaming query breaks if token contains backtick
(Rahul Goswami via Eric Pugh)
- SOLR-17197: Fix getting fieldType by its name in FileBasedSpellChecker
(Andrey Bozhko via Eric Pugh)
- SOLR-17198: AffinityPlacementFactory can fail if Shard leadership changes occur while it is collecting metrics.
(Paul McArthur)
- SOLR-17018: Add QueryLimits support to Learning To Rank rescoring.
(Alessandro Benedetti)
- SOLR-14892: Queries with shards.info and shards.tolerant can yield multiple null keys in place of shard names
(Mathieu Marie, David Smiley)
- SOLR-17110: Fixed JacksonJsonWriter to properly quote uuid values in json query response
(Andrey Bozhko via Eric Pugh)
- SOLR-17209: Fix NullPointerException in QueryComponent
(Vincent Primault via Eric Pugh)
- SOLR-17113: Correct how `/replication?command=details` describes errors in backup operations.
(Przemyslaw Ciezkowski via Christine Poerschke and Jason Gerlowski)
- SOLR-17200: Fix false positive race condition in `/health?requireHealthyCores=true` during core loading
(hossman)
- SOLR-17218: Fix indexFetcher logging to include MDC details
(hossman)
- SOLR-17206: Eliminate the possibility of a -1 status code for SolrCloud update requests that are distributed.
(Paul McArthur)
- SOLR-17213: Fix spurious warnings about solr url format in Solr CLI when users aren't providing a deprecated solr url.
(Eric Pugh)
- SOLR-17176: Fix log history V2 API serialization
(Michael Gibney)
- SOLR-11535: Fix race condition in singleton-per-collection StateWatcher creation
(Michael Gibney)
- SOLR-16866: Reorder nested directory deletions to avoid logging spurious NoSuchFileException
(Michael Gibney)
- SOLR-17194: Fix CloudHttp2SolrClient doesn't preserve the passed httpClient to Http2ClusterStateProvider when using the internalClientBuilder
(Eric Pugh, Lamine Idjeraoui, Houston Putman, Anshum Gupta)
- Dependency Upgrades (3)
- SOLR-17157: Upgrade Lucene to 9.10.0
(hossman, Christine Poerschke)
- SOLR-17214: Update forbiddenapis to 3.7
(hossman)
- GITHUB#2408: Update commons-cli to 1.7
(Eric Pugh)
- Other Changes (8)
- SOLR-17126: Cut over System.getProperty to EnvUtils.getProperty
(janhoy)
- SOLR-17066: GenericSolrRequest now has a `setRequiresCollection` setter that allows it to specify whether
it should make use of the client-level default collection/core.
(Jason Gerlowski)
- SOLR-17190: Replace org.apache.solr.util.LongSet with hppc LongHashSet
(Michael Gibney)
- SOLR-12089: Update FileBasedSpellChecker and IndexBasedSpellChecker to accept accuracy parameter
as float; deprecate `breakSugestionTieBreaker` parameter in favor of `breakSuggestionTieBreaker`
in WordBreakSolrSpellChecker
(Andrey Bozhko via Eric Pugh)
- SOLR-17201: Http2SolrClient and friends no longer marked as @lucene.experimental.
Krb5HttpClientBuilder and PreemptiveBasicAuthClientBuilderFactory no longer deprecated
(janhoy)
- SOLR-17217: Simplify verbose MatcherAssert.assertThat usage in tests
(hossman)
- SOLR-17232: PropertiesOutputStream is renamed IndexOutputOutputStream and overrides write(byte[], int, int).
(Bruno Roustant)
- SOLR-17222: QueryResponseWriterUtil.NonFlushingStream should support bulk write methods
(Michael Gibney)
- New Features (3)
- SOLR-17006: Collection creation & adding replicas: User-defined properties are persisted to state.json and
applied to new replicas, available for use as property substitution in configuration files.
(Vincent Primault)
- SOLR-16974: Circuit Breakers can now be configured globally
(janhoy, Christine Poerschke)
- SOLR-16743: When using TLS, Solr can now auto-reload the keystore and truststore without the need to restart the process.
This is enabled by default when running with TLS and can be disabled or configured in solr.in.sh
(Houston Putman, Tomás Fernández Löbbe)
- Improvements (24)
- SOLR-17053: Distributed search with shards.tolerant: if all shards fail, fail the request
(Aparna Suresh via David Smiley)
- SOLR-16924: RESTORECORE now sets the UpdateLog to ACTIVE state instead of requiring a separate
REQUESTAPPLYUPDATES call in Collection restore.
(Julia Maimone, David Smiley)
- SOLR-16907: Fail when parsing an invalid custom permission definition from security.json
(janhoy, Uwe Schindler)
- SOLR-13748: Add support for mm (min should match) parameter to bool query parser
(Andrey Bozhko)
- SOLR-17046: SchemaCodecFactory is now the implicit default codec factory.
(hossman)
- SOLR-16943: Extend Solr client tracing coverage to both Jetty Client and Apache HttpClient
(Alex Deparvu, David Smiley)
- SOLR-16397: Swap core v2 endpoints have been updated to be more REST-ful.
SWAP is now available at `POST /api/cores/coreName/swap`
(Sanjay Dutt via Jason Gerlowski)
- SOLR-17011: Add tracing spans to internal collection commands
(Alex Deparvu)
- SOLR-17041: Make CommitTracker currentTlogSize lazy
(Alex Deparvu)
- SOLR-16397: The rename-core v2 endpoint has been updated to be more REST-ful.
RENAME is now available at `POST /api/cores/coreName/rename`
(Sanjay Dutt via Jason Gerlowski)
- SOLR-17035: Add trace id to jetty thread names to improve debuggability via stack traces
(Alex Deparvu)
- SOLR-17079: Allow to declare replica placement plugins in solr.xml
(Vincent Primault)
- SOLR-16959: Make the internal CoreSorter implementation configurable in solr.xml
(Vincent Primault)
- SOLR-17050: Use compact JSON for Learning to Rank (LTR) feature and model storage.
(Florin Babes, Christine Poerschke, Alessandro Benedetti)
- SOLR-17094: Close objects contained inside an ObjectCache.
(Vincent Primault)
- SOLR-16577: Ensure core load failures are always logged.
(Haythem Khiri, David Smiley)
- SOLR-17063: Do not retain log param references in LogWatcher
(Michael Gibney)
- SOLR-17066: SolrClient builders now allow users to specify a "default" collection or core
using the `withDefaultCollection` method. Use of the Builder methods is preferable to including
the collection in the base URL accepted by certain client implementations.
(Jason Gerlowski)
- SOLR-15960: Unified use of system properties and environment variables
(janhoy)
- SOLR-16397: The MERGEINDEXES v2 endpoint has been updated to be more REST-ful.
MERGEINDEXES is now available at `POST /api/cores/coreName/merge-indices`
(Sanjay Dutt via Jason Gerlowski)
- PR#2186: Include the external file name in the log instead of the hard-coded value in FileFloatSource.java.
(Hamzeh Aldmour via Uwe Schindler)
- SOLR-17096: solr.xml now supports declaring clusterSingleton plugins
(Paul McArthur, David Smiley)
- SOLR-16397: The v2 endpoint to request the status of asynchronous CoreAdmin commands has been updated to be more REST-ful.
Now available at `GET /api/node/commands/someRequestId`
(Sanjay Dutt via Jason Gerlowski)
- SOLR-17068: bin/solr post CLI use of options is now aligned closely with bin/post CLI tool, and is consistently
referenced throughout the Ref Guide and source code, and is used through out our tests. The bin/post tool
remains and has been tested to work.
(Eric Pugh)
- Optimizations (2)
- SOLR-17084: LBSolrClient (used by CloudSolrClient) now returns the count of core tracked as not live AKA zombies
instead of the full list of cores. This list is potentially as long as the number of cores. When there are many
cores, this leads to high CPU and memory usage.
(Gilles Bellaton, David Smiley)
- SOLR-17036: UpdateLog lazy creates VersionBucket large array, and VersionBucket do not keep the highest version anymore.
This optimization reduces the memory usage, specially when the SolrCore is not used for indexing
(Bruno Roustant)
- Bug Fixes (11)
- SOLR-17045: DenseVectorField w/ vectorDimension > 1024 now work automatically with _default configset, due to
implicit use of SchemaCodecFactory.
(hossman)
- SOLR-10653: When there's a UUIDField in the schema and atomic update touches another field
the error occurs when leader updates replica
(Mikhail Khludnev)
- SOLR-17093: Collection restore API command now returns "requestid" when executed asynchronously like other APIs
(Tomás Fernández Löbbe)
- SOLR-16952: In distributed i.e. multi-shard cloud mode returned dense vector 'fl' fields are now a list of numbers instead of strings.
(Qais Qadri, Christine Poerschke)
- SOLR-17090: The v2 "delete alias" API no longer errantly returns a 405 status code
(Jason Gerlowski)
- SOLR-17121: Fix SchemaCodecFactory to get PostingsFormat and DocValues from field.
(Bruno Roustant, David Smiley)
- SOLR-17116: The INSTALLSHARDDATA "collection-admin" API now reports errors correctly when run asynchronously.
(Jason Gerlowski)
- SOLR-17074: Fixed not correctly escaped quote in bin/solr script
(Dominique Béjean, Vincenzo D'Amore)
- SOLR-17120: Fix NullPointerException in UpdateLog.applyOlderUpdates that can occur if there are multiple partial
updates of the same document in separate requests using commitWithin.
(Calvin Smith, Christine Poerschke)
- SOLR-17112: bin/solr script doesn't do ps properly on some systems.
(Vincenzo D'Amore via Eric Pugh)
- SOLR-17149: Backups on collections with too many shards fail due to restrictive Executor queue size
(Pierre Salagnac via Jason Gerlowski)
- Dependency Upgrades (25)
- PR#2055: Update org.mockito:mockito* from v5.5.0 to v5.8.0
(solrbot)
- PR#2165: Update org.eclipse.jetty*:* to v10.0.19
(solrbot)
- PR#2159: Update io.netty:* to v4.1.104.Final
(solrbot)
- PR#2048: Update dependency commons-cli:commons-cli to v1.6.0
(solrbot)
- PR#2174: Update org.slf4j:* to v2.0.10
(solrbot)
- PR#2135: Update dependency com.fasterxml.jackson:jackson-bom to v2.16.1
(solrbot)
- PR#2138: Update dependency org.apache.commons:commons-compress to v1.25.0
(solrbot)
- PR#2139: Update dependency org.apache.commons:commons-lang3 to v3.14.0
(solrbot)
- PR#2035: Update dependency com.google.errorprone:error_prone_annotations to v2.23.0
(solrbot)
- PR#2036: Update io.grpc:grpc-* to v1.59.0
(solrbot)
- PR#2037: Update org.apache.logging.log4j:* to v2.21.0
(solrbot)
- PR#2033: Update io.dropwizard.metrics:* to v4.2.21
(solrbot)
- PR#2032: Update dependency io.swagger.core.v3:swagger-annotations to v2.2.17
(solrbot)
- PR#2015: Update dependency org.codehaus.woodstox:stax2-api to v4.2.2
(solrbot)
- PR#2034: Update dependency com.github.spotbugs:spotbugs-annotations to v4.8.0
(solrbot)
- PR#2014: Update dependency com.google.guava:guava to v32.1.3-jre
(solrbot)
- PR#2010: Update dependency org.immutables:value-annotations to v2.10.0
(solrbot)
- PR#2009: Update dependency io.opentelemetry:opentelemetry-bom to v1.31.0
(solrbot)
- PR#2008: Update dependency org.semver4j:semver4j to v5.2.2
(solrbot)
- PR#1989: Update org.apache.zookeeper:* to v3.9.1
(solrbot)
- PR#1745: Update dependency com.google.cloud:google-cloud-bom to v0.204.0
(solrbot)
- SOLR-17012: Update Apache Hadoop to 3.3.6 and Apache Curator to 5.5.0
(Kevin Risden)
- SOLR-17026: Upgrade to Gradle 8.4
(Kevin Risden)
- SOLR-17089: Upgrade Jersey to 3.1.5
(Jason Gerlowski)
- SOLR-17097: Upgrade Lucene to 9.9.2
(Nazerke Seidan, Christine Poerschke, Pierre Salagnac)
- Other Changes (6)
- SOLR-17024: Remove support for the long-defunct "collectionDefaults" clusterprops key
(Jason Gerlowski)
- SOLR-17042: Deprecate the V2RequestSupport interface, and the associated `setUseV2` and `setUseBinaryV2` SolrRequest
methods. SolrJ users looking to make use of v2 APIs in their applications can use the SolrRequest implementations
dedicated to that purpose.
(Jason Gerlowski)
- SOLR-17072: package CLI tool prints error JSONPath
(Mikhail Khludnev)
- SOLR-17078: The `train_and_upload_demo_model.py` script referenced in LTR documentation now uses Python3
(Jason Gerlowski)
- SOLR-17091: dev tools script cloud.sh became broken after changes in 9.3 added a new -slim.tgz file it was not expecting
cloud.sh has been updated to ignore the -slim.tgz version of the tarball.
- SOLR-16880: Solr now produces an OpenAPI Specification artifact on releases ("solr-openapi-x.y.z.json") that covers
Solr's v2 APIs.
(Jason Gerlowski, Houston Putman)
- Bug Fixes (6)
- SOLR-17039: Entropy calculation in bin/solr script fails in Docker due to missing 'bc' cmd
(janhoy)
- SOLR-17057: JSON Query regression: If "query" is specified with a String (not JSON structure),
"defType" should parse it. Since 9.4 defType was ignored.
(David Smiley)
- SOLR-6853: Allow '/' characters in the text managed by Managed Resources API.
(Nikita Rusetskii via Eric Pugh)
- SOLR-17060: CoreContainer#create may deadlock with concurrent requests for metrics
(Alex Deparvu, David Smiley)
- SOLR-17098: ZK Credentials and ACLs are no longer sent to all ZK Servers when using Streaming Expressions.
They will only be used when sent to the default ZK Host.
(Houston Putman, Jan Høydahl, David Smiley, Gus Heck, Qing Xu)
- SOLR-16203: Properly initialize schema plugins loaded by SPI name
(janhoy, hossman, Uwe Schindler)
- Dependency Upgrades (22)
- PR#2165: Update org.eclipse.jetty*:* to v10.0.19
(solrbot)
- PR#2159: Update io.netty:* to v4.1.104.Final
(solrbot)
- PR#2048: Update dependency commons-cli:commons-cli to v1.6.0
(solrbot)
- PR#2174: Update org.slf4j:* to v2.0.10
(solrbot)
- PR#2135: Update dependency com.fasterxml.jackson:jackson-bom to v2.16.1
(solrbot)
- PR#2138: Update dependency org.apache.commons:commons-compress to v1.25.0
(solrbot)
- PR#2139: Update dependency org.apache.commons:commons-lang3 to v3.14.0
(solrbot)
- PR#2035: Update dependency com.google.errorprone:error_prone_annotations to v2.23.0
(solrbot)
- PR#2036: Update io.grpc:grpc-* to v1.59.0
(solrbot)
- PR#2037: Update org.apache.logging.log4j:* to v2.21.0
(solrbot)
- PR#2033: Update io.dropwizard.metrics:* to v4.2.21
(solrbot)
- PR#2032: Update dependency io.swagger.core.v3:swagger-annotations to v2.2.17
(solrbot)
- PR#2015: Update dependency org.codehaus.woodstox:stax2-api to v4.2.2
(solrbot)
- PR#2031: Update dependency com.fasterxml.jackson:jackson-bom to v2.15.3
(solrbot)
- PR#2034: Update dependency com.github.spotbugs:spotbugs-annotations to v4.8.0
(solrbot)
- PR#2014: Update dependency com.google.guava:guava to v32.1.3-jre
(solrbot)
- PR#2010: Update dependency org.immutables:value-annotations to v2.10.0
(solrbot)
- PR#2009: Update dependency io.opentelemetry:opentelemetry-bom to v1.31.0
(solrbot)
- PR#2008: Update dependency org.semver4j:semver4j to v5.2.2
(solrbot)
- PR#1989: Update org.apache.zookeeper:* to v3.9.1
(solrbot)
- PR#1743: SOLR-17012: Update Apache Hadoop to 3.3.6 and Apache Curator to 5.5.0
(solrbot)
- PR#1745: Update dependency com.google.cloud:google-cloud-bom to v0.204.0
(solrbot)
- Other Changes (1)
- SOLR-16949: Restrict certain file types from being uploaded to or downloaded from Config Sets
(janhoy, Houston Putman)
- New Features (6)
- SOLR-16654: Add support for node-level caches
(Michael Gibney)
- SOLR-16954: Make Circuit Breakers available for Update Requests
(janhoy, Christine Poerschke, Pierre Salagnac)
- SOLR-15056: A new Circuit breaker for percentage of CPU utilization is added. The former "CPU" circuit breaker
is now more correctly named LoadAverageCircuitBreaker as it trips on system load average which is not a percentage.
Users of legacy CircuitBreakerManager are not affected by this change.
(Walter Underwood, janhoy, Christine Poerschke, Atri Sharma)
- SOLR-15771: bin/auth creates reasonable roles and permissions for security: 'search', 'index', 'admin', and 'superadmin' and assigns user superadmin role.
(Eric Pugh, janhoy)
- SOLR-15367: Convert "rid" functionality into a default Tracer
(Alex Deparvu, David Smiley)
- SOLR-16852: Backups now allow metadata to be added as key-values
(Tomás Fernández Löbbe)
- Improvements (25)
- SOLR-16490: `/admin/cores?action=backupcore` now has a v2 equivalent, available at
`GET /api/cores/coreName/backups`
(Sanjay Dutt via Jason Gerlowski)
- SOLR-16883: Postlogs tool for indexing Solr logs in Solr now supported on Windows by converting it to a Solr CLI command: `bin/solr postlogs`. `bin/postlogs` script marked deprected.
(Eric Pugh, Will White)
- SOLR-16847: v2 APIs are now able to access any applicable solrconfig.xml "requestHandler" configuration.
(Alex Deparvu via Jason Gerlowski)
- SOLR-11685: When SolrCloud shard leaders change while indexing updates arrive, Solr could fail and return
a HTTP 503 status. Switched to 510 so that CloudSolrClient will auto-retry it and probably succeed.
(David Smiley, Alex Deparvu)
- SOLR-16490: The semi-internal `/admin/cores?action=restorecore` API now has a v2 equivalent, available at
`POST /api/cores/coreName/restore {...}`
(Sayanti Dey via Jason Gerlowski)
- SOLR-14667: Make zkClientTimeout consistent and based on a system property. The default values are stored in a single place referenced everywhere and they are based on system properties
(Alex Deparvu)
- SOLR-16926: The embedded Zookeeper's bind host can now be overridden, but still defaults to "127.0.0.1".
This is useful when using the ZkCli on a remote Solr using the embedded ZK, or Solr running in a Docker container.
The SOLR_ZK_EMBEDDED_HOST envVar or -Dsolr.zk.embedded.host sysProp control this bind address.
(Houston Putman)
- SOLR-16825: Solr now offers `SolrRequest` implementations for a subset of its v2 APIs. These implementations
are experimental and should be used with caution, but may be preferable to their v1 counterparts in some
circumstances as they are generated and more likely to remain up-to-date with future API changes.
(Jason Gerlowski, David Smiley, Houston Putman)
- SOLR-16927: Allow SolrClientCache clients to use Jetty HTTP2 clients
(Alex Deparvu, David Smiley)
- SOLR-16941: The SolrCLI now uses a smarter default for the Solr URL if none is provided, using the same envVars used when running Solr.
(Houston Putman)
- SOLR-16940: Users can pass Java system properties to the SolrCLI via the SOLR_TOOL_OPTS environment variable.
(Houston Putman)
- SOLR-15474: Make Circuit breakers individually pluggable
(Atri Sharma, Christine Poerschke, janhoy)
- SOLR-16982: Trip Circuit Breakers only for external requests
(janhoy, Christine Poerschke)
- SOLR-16896, SOLR-16897: Add support of OAuth 2.0/OIDC 'code with PKCE' flow
(Lamine Idjeraoui, janhoy, Kevin Risden, Anshum Gupta)
- SOLR-16879: Limit the number of concurrent expensive core admin operations by running them in a
dedicated thread pool. Backup, Restore and Split are expensive operations.
(Pierre Salagnac, David Smiley)
- SOLR-16964: The solr.jetty.ssl.sniHostCheck option now defaults to the value of SOLR_SSL_CHECK_PEER_NAME, if it is provided.
This will enable client and server hostName check settings to be governed by the same environment variable.
If users want separate client/server settings, they can manually override the solr.jetty.ssl.sniHostCheck option in SOLR_OPTS.
(Houston Putman)
- SOLR-16970: SOLR_OPTS is now able to override options set by the Solr control scripts, "bin/solr" and "bin/solr.cmd".
(Houston Putman)
- SOLR-16968: The MemoryCircuitBreaker now uses average heap usage over the last 30 seconds
(janhoy, Christine Poerschke)
- SOLR-14886: Suppress stack traces in query response
(Isabelle Giguere via Alex Deparvu)
- SOLR-16461: `/solr/coreName/replication?command=backup` now has a v2 equivalent, available at
`/api/cores/coreName/replication/backups`
(Sanjay Dutt, Jason Gerlowski, Alex Deparvu)
- SOLR-16938: Auto configure tracer without a <tracerConfig> tag in solr.xml
(Alex Deparvu)
- SOLR-16950: SimpleTracer propagation for manual transaction ids
- SOLR-15440: The Learning To Rank FieldValueFeature now uses DocValues when docValues=true and stored=true are combined.
(Christine Poerschke, Tom Gilke)
- SOLR-16959: Make the internal CoresLocator implementation configurable in solr.xml
(Vincent Primault via David Smiley)
- SOLR-16967: Some ConfigSet operations formerly required that solrconfig.xml exist but should not have because
the name of the file is configurable when creating cores / collections.
(David Smiley)
- Optimizations (4)
- SOLR-16845: BinaryResponseWriter should not attempt cast to Utf8CharSequence
(Alex Deparvu via Houston Putman)
- SOLR-16265: reduce memory usage of ContentWriter based requests in Http2SolrClient
(Alex Deparvu, Kevin Risden, David Smiley)
- SOLR-16989: Optimize and consolidate reuse of DocValues iterators for value retrieval
(Michael Gibney)
- SOLR-17004: ZkStateReader waitForState should check clusterState before using watchers
(Kevin Risden)
- Bug Fixes (34)
- SOLR-16886: Don't commit multi-part uploads that have been aborted
(Tomás Fernández Löbbe, Houston Putman)
- SOLR-16889: Rate Limiter should stop processing on 429
(Alex Deparvu, David Smiley)
- SOLR-16906: Correctly capture REPLICATION metrics in Prometheus config
(Daisuke Aritomo via Houston Putman)
- SOLR-16905: Allow access to specified "solr.allowPaths" in Security Manager
(daylicron, Houston Putman)
- SOLR-16922: Scripts wrongly prohibit embedded zookeeper when solr port is between 55535 and 64535
(Tiziano Degaetano, Colvin Cowie)
- SOLR-16360: Atomic update on boolean fields doesn't reflect when value starts with "1", "t" or "T"
(Rahul Goswami, Justin Sweeney, David Smiley)
- PR#1826: Allow looking up Solr Package repo when that URL references a raw repository.json hosted on Github when the file is JSON but the mimetype used is text/plain.
(Eric Pugh)
- SOLR-16944: V2 API /api/node/health should be governed by "health" permission, not "config-read"
(janhoy)
- SOLR-16859: Missing Proxy support for Http2SolrClient
(Alex Deparvu)
- SOLR-16929: SolrStream propagates undecoded error message
(Alex Deparvu)
- SOLR-16934: Allow Solr to read client (javax.net.ssl.*) trustStores and keyStores via SecurityManager.
(Houston Putman)
- SOLR-16946: Updated Cluster Singleton plugins are stopped correctly when the Overseer is closed.
(Paul McArthur)
- SOLR-16933: Include the full query response when using the API tool, and fix serialization issues for SolrDocumentList.
(Houston Putman)
- SOLR-16916: Use of the JSON Query DSL should ignore the defType parameter
(Christina Chortaria, Max Kadel, Ryan Laddusaw, Jane Sandberg, David Smiley)
- SOLR-16958: Fix spurious warning about LATEST luceneMatchVersion
(Colvin Cowie)
- SOLR-16955: Tracing v2 apis breaks SecurityConfHandler
(Alex Deparvu, David Smiley)
- SOLR-16044: SlowRequest logging is no longer disabled if SolrCore logger set to ERROR
(janhoy, hossman)
- SOLR-16415: asyncId must not have '/'; enforce this. Enhance ZK cleanup to process directories
instead of fail.
(David Smiley, Paul McArthur)
- SOLR-16899: CoreAdminOp are statically registered in CoreAdminHandler, preventing more than one Solr instance in the same JVM
(Vincent Primault via Alex Deparvu)
- SOLR-16963: The "solr.jetty.ssl.verifyClientHostName" sysProp and "SOLR_SSL_CLIENT_HOSTNAME_VERIFICATION" envVar have been fixed,
and the setting once again tells the server to check the originating client hostname against the client certificate when doing mTLS.
(Houston Putman, Tomás Fernández Löbbe)
- SOLR-16973: fix REMOTE_JMX_OPTS to delayed expansion
(Tiziano Degaetano via Colvin Cowie)
- SOLR-16971: RealTimeGet with Composite router throws NPE
(Alex Deparvu)
- SOLR-16931: ReRankScaler explain breaks with debug=true and in distributed mode
(Joel Bernstein)
- SOLR-16983: Fixed a bug that could cause some usages of SolrStream to fail to close InputStreams from the server.
Also fixed the usage of ObjectReleaseTracker in SolrTestCaseJ4 to catch these kinds of bugs
(hossman)
- SOLR-16925: Fix indentation for JacksonJsonWriter
(Houston Putman)
- SOLR-16701: Fix race condition on PRS enabled collection deletion
(Patson Luk)
- SOLR-16991: Concurrent requests failing JWT authentication in Admin UI intermittently
(Lamine Idjeraoui, janhoy)
- SOLR-16997: OTEL configurator NPE when SOLR_HOST not set
(janhoy)
- PR#1963: Fix the admin UI green core-size graph on nodes screen
(janhoy)
- SOLR-16980: Connect to SOLR standalone with basic authentication
(Alex Deparvu)
- SOLR-16992: Non-reproducible StreamingTest failures -- suggests CloudSolrStream concurency race condition
(Alex Deparvu, hossman)
- SOLR-16644: Fixing the entropy warning threshold using scaling based on poolsize
(Raghavan Muthuregunathan)
- SOLR-17009: json.wrf parameter ignored in JacksonJsonWriter
(Alex Deparvu)
- SOLR-17019: ZkCli should create subpaths when necessary
(Houston Putman)
- Dependency Upgrades (4)
- PR#1846: Update io.opentelemetry to 1.29.0
(Alex Deparvu)
- SOLR-16985: Upgrade Lucene to 9.8.0
(Alex Deparvu, Christine Poerschke)
- PR#1971: Update forbiddenapis to 3.6 to support Java 21/22 and commons-io up to 2.14.0
(Uwe Schindler)
- PR#2001: Update org.eclipse.jetty to 10.0.17
(Alex Deparvu)
- Other Changes (10)
- SOLR-16842: Eliminate special case code in Solr CLI by introducing explicit VersionTool.java.
(Eric Pugh, Will White)
- SOLR-16856: The default auto-soft-commit time has been set to 3 seconds. Previously, auto-soft-commit was disabled by default.
(Houston Putman)
- PR#1629: Fix typos in org.apache.solr.core package
(Andrey Bozhko, Marcus Eagan)
- SOLR-16803: Remove dependencies on methods defined in the SimplePostTool from Solr core and solrj modules.
- SOLR-16915: Lower the AffinityPlacementPlugin's default minimalFreeDiskGB to 5 GB
(Houston Putman)
- SOLR-6994: Implement Windows version of bin/post via implementing bin/solr post command that works on Windows and Unix. Deprecate bin/post.
(Eric Pugh, Will White)
- SOLR-16623: new SolrJettyTestRule for tests needing HTTP or Jetty.
(David Smiley, Joshua Ouma)
- SOLR-16979: BATS integration tests now start solr instances on a randomly selected port
(janhoy)
- SOLR-16978: Be case insensitive when parsing booleans from text
(Tomás Fernández Löbbe)
- SOLR-16960: Tests should sometimes run with a Tracer (not no-op)
(Alex Deparvu)
- Upgrade Notes (1)
- SOLR-13396: Add property to control the deletion of unknown cores. The default setting is to not delete unknown cores.
This is a change to behaviour introduced in Solr 7.3.1 by SOLR-12066. To continue using the previous behaviour, Solr can be
started with the system property `solr.deleteUnknownCores` set to `true`.
(Colvin Cowie, janhoy)
- New Features (14)
- SOLR-16697: Solr now provides an "Install Shard" API that allows users who have built (per-shard) indices offline to import
them into SolrCloud shards. The API is available at `POST /api/collections/collName/shards/shardName/install`
(v2), or at `GET /solr/admin/collections?action=INSTALLSHARDDATA` (v1).
(Jason Gerlowski)
- SOLR-16742: Produce a slim Solr binary and docker image, without the Prometheus Exporter and Solr Modules.
(Houston Putman)
- SOLR-16674: Introduced support for byte vector encoding in DenseVectorField and KnnQParser
(Elia Porciani via Alessandro Benedetti).
- SOLR-16719: AffinityPlacementFactory now supports spreading replicas across domains within the availability zone and
optionally fail the request if more than a configurable number of replicas need to be placed in a single domain.
(Houston Putman, Tomás Fernández Löbbe)
- SOLR-16836: Introduced support for high dimensional vectors
(Alessandro Benedetti).
- SOLR-16806: Solr now provides a BalanceReplicas API at `POST /api/cluster/replicas/balance` (v2), to spread replicas
across a given set of nodes. No v1 API is available.
(Houston Putman, Tomás Fernández Löbbe, Jason Gerlowski, Radu Gheorghe)
- SOLR-16827: Add min/max scaling to the reranker
(Joel Bernstein)
- SOLR-16844: Added new parameter `backupConfigset` to collection Backup to optionally skip backing up configsets.
(Tomás Fernández Löbbe)
- SOLR-16837: Introducing AffinityPlacementFactory.withCollectionShards to collocate corresponding shards of two collections.
eg primaryColl.shard1 will be placed to the node where secondaryColl.shard1 resides, etc.
(Mikhail Khludnev)
- SOLR-16812: Support CBOR format for update/query
(noble)
- SOLR-16855: Solr now provides a MigrateReplicas API at `POST /api/cluster/replicas/migrate` (v2), to move replicas
off of a given set of nodes. This extends the functionality of the existing ReplaceNode API.
(Houston Putman)
- SOLR-16717: {!join} can join collections with multiple shards on both sides.
(Mikhail Khludnev)
- SOLR-16675: Added function queries for dense vector similarity.
(Elia Porciani, Alessandro Benedetti)
- SOLR-16884: Solr Javadocs now have searching enabled.
(Mike Drob, Houston Putman)
- Improvements (42)
- SOLR-16183: XML update allows <doc> <doc name="child"> ... </doc> </doc>.
(Vinayak Hegde via Mikhail Khludnev)
- SOLR-16638: Fix Http2SolrClient's exception message when serverBaseUrl is null
(Alex Deparvu via Kevin Risden)
- SOLR-16393: The path of the v2 "delete alias" API has been tweaked slightly to be more intuitive, and is now available at
`DELETE /api/aliases/aliasName`.
(Jason Gerlowski)
- SOLR-16393: Solr's v2 "aliasprop" CRUD APIs have been tweaked slightly to be more intuitive. Alias property modification
is now available at `PUT /api/aliases/aliasName/properties` (for bulk modification) and `PUT /api/aliases/aliasName/properties/propName`
(for single property updates). Additionally new APIs have been added for listing properties (`GET /api/aliases/aliasName/properties`),
fetching single property values (`GET /api/aliases/aliasName/properties/propName`), and property deletion
(`DELETE /api/aliases/aliasName/properties/propName`).
(Alex Deparvu via Jason Gerlowski)
- SOLR-10466: Introduce Builder setter for defaultCollection on cloud SolrClients. Deprecated
direct setter setDefaultCollection on cloud SolrClients.
(Eric Pugh, David Smiley)
- SOLR-16391: The path of the v2 "collprop" API has been tweaked slightly to be more intuitive. Endpoints are now
available under the `PUT` and `DELETE` verbs at `/api/collections/collName/properties/propName` depending on whether the property is
being upserted or deleted.
(Jason Gerlowski)
- SOLR-16394: The path of the v2 "collection backup" API has been tweaked slightly to be more intuitive, and is now available at
`POST /api/collections/backups/backupName/versions`.
(Jason Gerlowski)
- SOLR-16458: Solr's v2 "logging" APIs have been tweaked slightly to be more intuitive. The single v2 endpoint that existing previously
(GET /api/node/logging) has been split into several different APIs. Logger levels can be fetched or modified at
`GET /api/node/logging/levels` and `PUT /api/node/logging/levels`, respectively. Log messages can be returned from a "listener" at
`GET /api/node/logging/messages`. And the threshold for the log "listener" can be modified with
`PUT /api/node/logging/messages/threshold`.
(Jason Gerlowski, Calvince Otieno)
- SOLR-16504: Convert CLI tools to use Jetty HTTP 2 client.
(Bence Szabo via Eric Pugh)
- SOLR-16752: Docker: Leave out optional apt packages, slightly reducing image size and lowering attack surface
(Quang-Cuong Bui)
- SOLR-16711: Extract SolrCLI tool implementations into their own classes and reorganize CLI related classes into new org.apache.solr.cli package.
(Eric Pugh)
- SOLR-15493: Throw an error message if the feature store doesn't exist.
(Ilaria Petreti via Alessandro Benedetti)
- SOLR-15737: Solr's collection-level "snapshot" APIs now have v2 equivalents. Snapshots can be created at `POST
/api/collections/collName/snapshots/snapshotName`, listed at `GET /api/collections/collName/snapshots`, and deleted at
`DELETE /api/collections/collName/snapshots/snapshotName`.
(John Durham via Jason Gerlowski)
- SOLR-16391: Solr's v2 "create collection" API has been tweaked slightly to be more intuitive. It remains available under
`POST /api/collections`, but the top level "create" command-name key has been removed.
(Jason Gerlowski)
- SOLR-16752: Docker: Leave out optional apt packages, slightly reducing image size and lowering attack surface
(Quang-Cuong Bui)
- SOLR-16711: Extract SolrCLI tool implementations into their own classes and reorganize CLI related classes into new org.apache.solr.cli package.
(Eric Pugh)
- SOLR-15493: Throw an error message if the feature store doesn't exist.
(Ilaria Petreti via Alessandro Benedetti)
- SOLR-16393: The v2 "create alias" API has been tweaked to be more intuitive. The format of the request body has changed
slightly: the "create-alias" command specifier has been removed, the "name" field for individual routers has been renamed
to "type", and the routers themselves are now always grouped into a list. Additionally the v2 API has moved to the new path
`POST /api/aliases`.
(Jason Gerlowski)
- SOLR-16394: The v2 "restore" API has been tweaked to be more intuitive. The top-level "restore-collection" command
specifier has been removed, and the API now lives at the new path `POST /api/backups/backupName/restore`.
(Jason Gerlowski)
- SOLR-16782: bin/solr export tool now supports JSON export format as well as the existing JSON With Lines format.
(Eric Pugh)
- SOLR-16392: The v2 deletereplica API has been tweaked to be more intuitive: single replicas deletion now uses
`DELETE /api/collections/cName/shards/sName/replicas/rName`, deletion of multiple relicas in a single shard uses
`DELETE /api/collections/cName/shards/sName/replicas?count=123`, and deletion of multiple replicas from all shards now uses
`PUT /api/collections/cName/scale {"count": 123"}`
(Jason Gerlowski)
- SOLR-14853: Security: Converted enableRemoteStreaming and enableStreamBody solrconfig options into system properties and env vars.
Attempts to set them the old way are no-op and log a warning.
(David Smiley, janhoy, Ishan Chattopadhyaya)
- SOLR-16766: Remove -usage as option from bin/solr, and ensure both -help and -h work.
(Bence Szabo via Eric Pugh)
- SOLR-16759: Introducing logAll parameter in the feature logger
(Anna Ruggero, Alessandro Benedetti)
- SOLR-9378: Internal shard requests no longer include the wasteful shard.url param. [shard] transformer now defaults to returning
only the shard id (based on luceneMatchVersion), but can be configured to return the legacy list of replicas.
(hossman)
- SOLR-16394: The v2 list and delete (collection) backup APIs have been tweaked to be more intuitive: backup listing now uses
`GET /api/backups/bName/versions`, backup deletion by ID now uses `DELETE /api/backups/bName/versions/1`, backup deletion by
recency now uses `DELETE /api/backups/bName/versions?retainLatest=3`, and index-file "garbage collection" now uses
`PUT /api/backups/backupName/purgeUnused`
(Jason Gerlowski)
- SOLR-16470: `/coreName/replication?commit=indexversion` now has a v2 equivalent, available at
`GET /api/cores/coreName/replication/indexversion`
(Matthew Biscocho via Jason Gerlowski)
- SOLR-16687: Add support of SolrClassLoader to SolrZkClient
(Lamine Idjeraoui via Jason Gerlowski & Houston Putman)
- SOLR-9378: Internal shard requests no longer include the wasteful shard.url param. [shard] transformer now defaults to returning
only the shard id (based on luceneMatchVersion), but can be configured to return the legacy list of replicas.
(hossman)
- SOLR-16816: Update node metrics while making affinityPlacement selections. Therefore selections can be made given the expected cluster
information after the previous selections are implemented.
(Houston Putman)
- SOLR-16392: The v2 "create shard" API has been tweaked to be more intuitive, by removing the top-level "create"
command specifier. The rest of the API remains unchanged.
(Jason Gerlowski)
- SOLR-16806: The included PlacementPlugins (Random, Simple, MinimizeCores and Affinity) now all implement the
OrderedNodePlacementPlugin, which provides the implementation for computePlacements() and computeBalancing().
Each implementing PlacementPlugin provides a way of weighting Solr Nodes, and the OrderedNodePlacement plugin
then uses the weights to decide the optimal strategy for placing new replicas or balancing existing replicas.
(Houston Putman, Tomás Fernández Löbbe, Jason Gerlowski, Radu Gheorghe)
- SOLR-16392: The v2 "add-replica" API has been tweaked to be more intuitive, by removing the top-level command specifier and
changing the path. The v2 functionality can now be accessed at: `POST /api/collections/cName/shards/sName/replicas {...}`
(Jason Gerlowski)
- SOLR-16398: Several v2 "command" APIs have been tweaked to be more intuitive. "FORCELEADER" is now available at
`POST /api/collections/cName/shards/sName/force-leader`, "RELOAD" is now available at `POST /api/collections/cName/reload`,
"SYNCSHARD" is now available at `POST /api/collections/cName/shards/sName/sync-shard`, and "RENAME" is now available at
`POST /api/collections/cName/rename`.
(Jason Gerlowski)
- SOLR-16398: The v2 "balance-shard-unique" API has been tweaked to be more intuitive, by removing the top-level command
specifier from the request body, and changing the path. The v2 functionality can now be accessed at:
`POST /api/collections/cName/balance-shard-unique {...}`
(Jason Gerlowski)
- SOLR-16806: The included PlacementPlugins (Random, Simple, MinimizeCores and Affinity) now all implement the
OrderedNodePlacementPlugin, which provides the implementation for computePlacements() and computeBalancing().
Each implementing PlacementPlugin provides a way of weighting Solr Nodes, and the OrderedNodePlacement plugin
then uses the weights to decide the optimal strategy for placing new replicas or balancing existing replicas.
(Houston Putman, Tomás Fernández Löbbe, Jason Gerlowski, Radu Gheorghe)
- SOLR-16820: Align allowed collection names and the validation of them in the CreateCollectionAPI and the PackageTool.
(Will White via Eric Pugh)
- SOLR-16748: The Official Solr Dockerfile has been cleaned up. The SOLR_DOWNLOAD_SERVER arg is now the only way to
specify where to download Solr from. If the url includes "apache.org", then GPG checks will be computed, otherwise
the GPG checks will be skipped.
(Houston Putman)
- SOLR-16470: `/coreName/replication?command=filelist` now has a v2 equivalent, available at
`GET /api/cores/coreName/replication/files?generation=123`
(Matthew Biscocho via Jason Gerlowski)
- SOLR-16878: Use Log4J JUL manager when starting Java. This is necessary for Lucene logs to be included with Solr logs.
(Houston Putman, Uwe Schindler)
- SOLR-16397: Reload core v2 endpoints have been updated to be more REST-ful.
RELOAD is now available at `POST /api/cores/coreName/reload`
(Sanjay Dutt via Jason Gerlowski)
- SOLR-16397: Unload core v2 endpoints have been updated to be more REST-ful.
UNLOAD is now available at `POST /api/cores/coreName/unload`
(Sanjay Dutt via Jason Gerlowski)
- Optimizations (8)
- SOLR-16694: Shortcut Http2SolrClient mimeType check for default response parse
(Tomás Fernández Löbbe, Alex Deparvu)
- SOLR-16692: Allow setting the a CookieStore implementation in Http2SolrClient. When Cookies are not needed, this can be set to
`org.eclipse.jetty.util.HttpCookieStore.Empty`, which avoids some of the processing that happens in the default `java.net.InMemoryCookieStore`.
Solr can also be started with the system property `solr.http.disableCookies` set to `true`, so that the internal clients
that Solr uses to communicate between nodes use `HttpCookieStore.Empty`.
(Tomás Fernández Löbbe)
- SOLR-16693: For query timeAllowed, switch from ExitableDirectoryReader to TimeLimitingBulkScorer
(David Smiley)
- SOLR-16764: Clarify that ExportTool exports documents in JSON with Lines format, not standard JSON. Add explicit -compress option for gzipping output.
Add ability to specific a directory for output along with a specific file when using -out.
(Eric Pugh)
- SOLR-14630: When querying with the `_route_` or `shards.preference` parameter and if there are multiple replicas for a
collection on the receiving node, there was sometimes an extra HTTP hop or the non-intended replica may receive the
request. SolrJ will now form the URL directly to the intended replica of the collection (and not the collection itself).
(Pierre Salagnac, Ivan Djurasevic, Jason Baik)
- SOLR-16846: Use IndexOrDocValuesQuery for PointFields on Set queries. This improves the performance in some scenarios when
the set query is combined with other selective queries.
(Tomás Fernández Löbbe, Alex Deparvu)
- SOLR-16869: Enable Incubating Java Panama Vector APIs when using Java 20 or 21.
When run using compatible hardware, Vector search will see considerable speed up.
(Houston Putman, Uwe Schindler)
- SOLR-16273: Improve the performance of the Prometheus Metric Exporter scraping metrics
(Matthew Biscocho via David Smiley & Christine Poerschke)
- Bug Fixes (27)
- SOLR-16638: Fix Http2SolrClient's exception message when serverBaseUrl is null
(Alex Deparvu via Kevin Risden)
- SOLR-16723: Http2SolrClient should not use Apache Http client classes
(Kevin Risden)
- SOLR-9775: NPE in QueryResultKey when running clustering search query in some cases.
(Roman Kagan, Christine Poerschke via Eric Pugh)
- SOLR-15083: Fix default solr-exporter-config.xml metrics with incorrect time unit conversions.
(Matthew Biscocho via janhoy)
- SOLR-16773: UI: Cloud>Nodes screen - fix display of cores with non-standard shard names
(janhoy, hossman)
- SOLR-7609: Internal update requests should fail back to the client in some edge cases for shard splits. Use HTTP status 510 so the client can retry the operation.
(Alex Deparvu, David Smiley, Tomás Fernández Löbbe)
- SOLR-16737: Http2SolrClient needs to inherit all properties when initialized with another http2 client
(Alex Deparvu, Tomás Fernández Löbbe)
- SOLR-13605: Fix setting client scoped socket and connect timeouts when using HttpSolrClient.Builder.withHttpClient() method.
(Eric Pugh, Alex Deparvu)
- SOLR-16777: Schema Designer now correctly manages trust of the ConfigSets it is managing.
(Ishan Chattopadhyaya, Skay, Houston Putman)
- SOLR-16771: Fixed behavior and handling of 'unset' logging levels in /admin/info/logging API and related Admin UI
(hossman)
- SOLR-16783: Fixed race condition deleting empty `clusterstate.json` file that could prevent Solr 9 instances from starting with a NoNodeException
(Tomás Fernández Löbbe)
- SOLR-16789: Fixed Admin UI dashboard "Security" panel to show the right information regarding authentication
(Tomás Fernández Löbbe)
- SOLR-16760: bin/solr package tool supports -h and -help command like other tools.
(Eric Pugh)
- SOLR-16765: bin/solr export tool limit property was off by 1. Now limit results exported to the exact number.
(Eric Pugh)
- SOLR-16801: Reset the thread's contextClassloader after loading the CoreContainer
(Thomas Wöckinger, Houston Putman)
- SOLR-16808: Stop publishing environment variables via the Metrics API
(Houston Putman)
- SOLR-16811: /update/json/docs?echo and mapUniqueKeyOnly params doesn't work
(Mikhail Khludnev)
- SOLR-16807: DenseVectorField breaks catch-all copyFields
(Joel Bernstein)
- SOLR-16831: Fixed bin/solr healthcheck command returning cluster status details. Removed SolrCloudTool class that is no longer used.
(Eric Pugh)
- SOLR-16848: Flaky DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
(Alex Deparvu, Paul McArthur)
- SOLR-16619: Fix solr scripts running on IBM i
(Jesse Gorzinski via Eric Pugh)
- SOLR-16809: The configuration for hiding sensitive sysProp information has been joined under `-Dsolr.hiddenSysProps` and `SOLR_HIDDEN_SYS_PROPS`.
(Houston Putman, David Smiley)
- SOLR-16860: Coordinator ZkStateReader's watch is not removed upon collection deletion
(Patson Luk)
- SOLR-16861: Coordinator node does not have the correct collection/core in MDCLoggingContext
(Patson Luk)
- SOLR-16877: BackupManager now allows ConfigSet "files" to be empty, which previously caused NPEs.
(Paul Blanchaert via Houston Putman)
- SOLR-16753: PRS state is now always updated at the end of a splitShard.
(Houston Putman, hossman)
- SOLR-16891: DocCollection, Slice and Replica now write to Json and write to Maps identically.
(Houston Putman, Tomás Fernández Löbbe)
- Dependency Upgrades (26)
- PR#1744: Update software.amazon.awssdk:* to v2.20.97
(solrbot)
- PR#1712: Update dependency biz.aQute.bnd:biz.aQute.bnd.annotation to v6.4.1
(solrbot)
- PR#1713: Update dependency io.swagger.core.v3:swagger-annotations to v2.2.12
(solrbot)
- PR#1715: Update io.grpc:grpc-* to v1.56.0
(solrbot)
- PR#1714: Update dependency org.xerial.snappy:snappy-java to v1.1.10.1
(solrbot)
- PR#1611: Update dependency org.mockito:mockito-core to v5
(solrbot)
- PR#1606: Update dependency com.google.cloud:google-cloud-bom to v0.197.0
(solrbot)
- PR#1641: Update dependency org.carrot2:carrot2-core to v4.5.1
(solrbot)
- PR#1686: Update dependency com.google.guava:guava to v32
(solrbot)
- PR#1584: Update io.netty:* to v4.1.93.Final
(solrbot)
- PR#1582: Update dependency com.github.ben-manes.caffeine:caffeine to v3.1.6
(solrbot)
- PR#1685: Update io.dropwizard.metrics:* to v4.2.19
(solrbot)
- PR#1599: Update dependency com.fasterxml.jackson:jackson-bom to v2.15.2
(solrbot)
- PR#1532: Update dependency joda-time:joda-time to v2.12.5
(solrbot)
- PR#1529: Update dependency com.jayway.jsonpath:json-path to v2.8.0
(solrbot)
- PR#1521: Update dependency org.hsqldb:hsqldb to v2.7.1
(solrbot)
- PR#1530: Update dependency org.apache.commons:commons-configuration2 to v2.9.0
(solrbot)
- PR#1515: Update org.apache.hadoop:* to v3.3.5
(solrbot)
- PR#1493: Update dependency org.apache.commons:commons-compress to v1.23.0
(solrbot)
- PR#1473: Update org.apache.calcite to v1.34.0
(solrbot)
- PR#1472: Update Jetty to 10.0.14, slf4j to 2.0.7, io.dropwizard.metrics to v4.2.18
(solrbot)
- PR#1451: Update dependency org.semver4j:semver4j to v4.3.0
(solrbot)
- PR#1494: Upgrade forbiddenapis to 3.5
(Uwe Schindler)
- SOLR-16642: Upgrade to Lucene 9.5.0
(Vinayak Hegde, David Smiley)
- SOLR-16799: upgrade Solr to use Lucene 9.6.0
(Alex Deparvu, Tomás Fernández Löbbe)
- SOLR-16853: Upgrade Lucene to 9.7.0
(Alex Deparvu)
- Other Changes (12)
- SOLR-16704: Add simple search benchmark to benchmark module
(Tomás Fernández Löbbe)
- SOLR-16709: Update Solr's internal use of Solr Client Builders to use TimeUnit parameter for time specific builder methods.
(Eric Pugh)
- SOLR-16713: Replace Guava usages with pure Java
(Kevin Risden)
- SOLR-16716: Replace commons-io usages with pure Java
(Kevin Risden)
- SOLR-16715: Replace new HashMap(int) and new HashSet(int) with CollectionUtil.newHashMap / CollectionUtil.newHashSet
(Kevin Risden)
- SOLR-16507: Change SplitShardCmd to not use NodeStateProvider
(Vinayak Hegde, David Smiley)
- SOLR-15703: replace all SolrException.log usage in Solr to just call log.error(...) directly
(Kevin Risden)
- SOLR-16604: Use Solr Client Builders directly in unit tests instead of delegating to SolrTestCaseJ4.
(Eric Pugh, David Smiley)
- PR#1567: Remove deprecated constructor in MetricsMap
(janhoy)
- SOLR-16745: Consolidate DelegatedSolrQueryRequest and DelegatingSolrQueryRequest
(Kevin Risden)
- PR#1566: Removed deprecated members from Tuple class
(janhoy)
- SOLR-16841: Added QueryResponseWriter benchmarks
(Alex Deparvu via Tomás Fernández Löbbe)
- Bug Fixes (9)
- SOLR-16731: Use the right cluster property for displaying if TLS is enabled
(Tomás Fernández Löbbe)
- SOLR-16730: Fix NPE in SystemInfoHandler for inter-node requests that would cause the Nodes page not to load.
SystemInfoHandler no longer populates the username, roles and permissions in inter-node requests.
(Tomás Fernández Löbbe)
- SOLR-16728: Fix Classloading Exception for inter-node requests when using SSL and HTTP2.
All Jetty classes are able to be shared between the Jetty server and webApp now.
(Houston Putman)
- SOLR-16734: SOLR_DATA_HOME is only honored in verbode mode
(Kevin Risden)
- SOLR-16721: Java version detection fails when `_JAVA_OPTIONS` is set
(janhoy, Ruoyu Zhong, Andy Webb)
- SOLR-16649: Http2SolrClient.processErrorsAndResponse uses wrong instance of ResponseParser
(Andrzej Białecki)
- SOLR-16240: Fix KerberosPlugin module classloading when using the hadoop-auth module.
Plugins in modules/packages that require the Thread contextClassLoader on startup should now work.
(Houston Putman)
- SOLR-16755: bin/solr's '-noprompt' option no longer works for examples
(hossman, janhoy, Houston Putman)
- SOLR-16741: CLUSTERSTATUS API returns wrong value for state ,leader for PRS collections
(noble)
- Other Changes (2)
- SOLR-16735: Allow custom configuration of Jetty's SecureRequestCustomizer
(Tomás Fernández Löbbe)
- SOLR-16747: The Solr source release can now be built/checked using gradle.
The solr-ref-guide tasks will not be included by default if the root directory is not a github repository.
(Houston Putman)
- Dependency Upgrades (1)
- (No changes)
- New Features (9)
- SOLR-16435: Add Request timeout to Http2SolrClient
(Tomás Fernández Löbbe)
- SOLR-16500: Added --enable-preview for JDK 19 to enable MemorySegment support in MMapDirectory
(Kevin Risden)
- SOLR-16496: QueryElevationComponent now supports fq exclusions
(Rudi Seitz via David Smiley)
- SOLR-16409: Admin UI - Expose all highlighting parameters in the Query UI
(Jeanie Lam via Eric Pugh)
- SOLR-16532: New OpenTelemetry (OTEL) module with OTLP/gRPC trace exporter. See ref.guide.
(janhoy, David Smiley)
- SOLR-16596: Learning To Rank - Added support for null feature values in multiple additive trees models
(Anna Ruggero via Alessandro Benedetti)
- SOLR-16608: Ability to compress state.json in Zookeeper
(Justin Sweeney via noble)
- SOLR-16646: New function query operator isnan to verify if value is NaN
(Gabriel Magno via Kevin Risden)
- SOLR-16643: Add reRankOperator=multiply/replace options to rerank query parser
(Andy Webb, Christine Poerschke, Mikhail Khludnev)
- Improvements (35)
- SOLR-16428: IgnoreLargeDocumentProcessorFactory now supports a "permissive" mode, where it logs and skips
offending documents but doesn't short-circuit the entire batch or return a 4xx error. This mode can be enabled
by setting the `permissiveMode` boolean option to `true` in your solrconfig.xml's
IgnoreLargeDocumentProcessorFactory declaration.
(Jason Gerlowski)
- SOLR-16392: A number of v2 "replica" endpoints have been updated to be more REST-ful, and hopefully, intuitive.
ADDREPLICAPROP is now `PUT /api/collections/$coll/shards/$shard/replicas/$rep/properties/$prop {"value": $val}`
DELETEREPLICAPROP is now `DELETE /api/collections/$coll/shards/$shard/replicas/$rep/properties/$prop`
- SOLR-11657: Deprecate ContentStream in favour of RequestWriter.ContentWriter.
(Joshua Ouma via Eric Pugh)
- SOLR-16302: WARN when restoring backup if ConfigSet with same name already exists
(Albert Moser via Kevin Risden)
- SOLR-8803: Java will now crash the Solr process on OOME and create a crash file that logs the
cause. This capability is now also present on Windows. Before, with bin/solr but not
bin/solr.cmd, Solr would execute a script that killed the Solr pid, leaving a very small time
window where Solr would continue to execute in an unpredictable state.
(Shawn Heisey, Kevin Risden)
- SOLR-15748: A v2 equivalent of CLUSTERSTATUS command is now available at `GET /api/cluster`. Collection listing,
previously at this path, can still be accessed at `GET /api/collections`.
(Joshua Ouma via Jason Gerlowski)
- SOLR-16475: Make the default replica placement plugin configurable as system property on startup
(janhoy)
- SOLR-13626: Document the SystemInfoHandler in the Ref Guide.
(Tony Cook via Eric Pugh)
- SOLR-16420: Introducing `{!mlt_content}foo bar` to cover existing `/mlt` handler functionality for SolrCloud.
(Mikhail Khludnev)
- SOLR-11028: A v2 equivalent of the `/admin/collections?action= REPLACE` command is now available at
`POST /api/cluster/nodes/nodeName/replace`.
(Joshua Ouma via Jason Gerlowski)
- SOLR-15479: A v2 equivalent of the RENAME command is now available at `POST /api/collections/collName/rename`.
(Anakhe Ajayi via Jason Gerlowski)
- SOLR-16158: Certain Jetty "server" jars (jetty-util, jetty-io, etc.) are no longer duplicated between server/lib and WEB-INF/lib. All shared jars now live in server/lib/ext.
By default, Jetty does not allow these jars to be shared by the Jetty server and the web-app (Solr).
In order to accomplish this, explicit exclusions have been added to server/contexts/solr-jetty-context.xml that allow these "server" jars to be shared.
Solr Core also no longer contains the JettySolrRunner (now living in the testing framework), removing the need for Jetty Server dependencies in solr-core.
(Houston Putman)
- SOLR-16565: posting the same file to the package store should not throw an error
(noble)
- SOLR-8975: Introduce Builder setters for setters that exist on SolrClients for responseParser, requestWriter.
Deprecated these SolrClient setters useMultiPartPost, and followRedirects.
(Eric Pugh, David Smiley, Kevin Risden)
- SOLR-16438: Support optional split.setPreferredLeaders prop in shard split command.
(Bruno Roustant)
- SOLR-10463: Introduce Builder setter for retryExpiryTime on cloud SolrClients. Deprecated
direct setter setRetryExpiryTime on cloud SolrClients.
(Eric Pugh)
- SOLR-10461: Introduce Builder setter for aliveCheckInterval on load balanced SolrClients. Deprecated
direct setter setAliveCheckInterval on SolrClients.
(Eric Pugh, David Smiley, Alex Deparvu)
- SOLR-10462: Introduce Builder setter for pollQueueTime on ConcurrentUpdateHttp2SolrClient. Deprecated
direct setter setPollQueueTime on ConcurrentUpdateHttp2SolrClient.
(Eric Pugh)
- SOLR-10464: Introduce Builder setter for collectionCacheTtl on cloud SolrClients. Deprecated
direct setter setCollectionCacheTTL on cloud SolrClients.
(Eric Pugh, David Smiley)
- SOLR-10452: Introduce Builder setter withTheseParamNamesInTheUrl for queryParams, renaming them to urlParamNames
to clarify they are parameter names, not the values. Deprecated direct setter setQueryParams and addQueryParams
on SolrClients.
(Eric Pugh, David Smiley, Alex Deparvu)
- SOLR-10470: Introduce Builder setter for parallelCacheRefreshes on cloud SolrClients. Deprecated
direct setter setParallelCacheRefreshes on cloud SolrClients.
(Eric Pugh, David Smiley, Alex Deparvu)
- SOLR-16616: JWTAuthPlugin: Read trusted X509 certificates from multiple files
(janhoy)
- SOLR-15772: More visible security warnings in Admin UI
(janhoy)
- SOLR-15787: FileSystemConfigSetService: implement the abstraction completely. It could be useful
for putting ConfigSets on a shared file system.
(Nazerke Seidan, David Smiley)
- SOLR-6312: SolrJ Cloud clients now correctly support isUpdatesToLeaders() being false. This behavior can also be
configured per AbstractUpdateRequest instance.
(hossman)
- SOLR-16635: Resolve crypto keys file path in MiniSolrCloudCluster. This eliminates the need to set the system
properties `pkiHandlerPublicKeyPath` and `pkiHandlerPrivateKeyPath` before starting MiniSolrCloudCluster when using
the Solr test framework.
(Tomás Fernández Löbbe)
- SOLR-16618: Admin UI Analysis page should include dynamic fields
(Alex Deparvu via Eric Pugh)
- SOLR-16590: Standardize Builder method names on SolrClient's to use the with pattern, deprecating the use of set or bare property name.
(Eric Pugh)
- SOLR-16595: Standardize Builder methods handling of times to use require a TimeUnit to be passed in. Deprecated methods that
do not specify a TimeUnit.
(Eric Pugh)
- SOLR-16665: The base docker image has been upgraded from Ubuntu 20 (Focal) to Ubuntu 22 (Jammy).
(Houston Putman)
- SOLR-11029: A v2 equivalent of the `/admin/collections?action=DELETENODE` command is now available at
`POST /api/cluster/nodes/nodeName/clear`.
(Bence Szabo via Jason Gerlowski)
- SOLR-16462: v2 equivalents of the "Core Admin" `LISTSNAPSHOT`, `CREATESNAPSHOT`, and `DELETESNAPSHOT` commands are now available at
`GET /api/cores/coreName/snapshots`, `POST /api/cores/coreName/snapshots/snapshotName`, and
`DELETE /api/cores/coreName/snapshots/snapshotName`, respectively
(John Durham via Jason Gerlowski)
- SOLR-16488: The paths of the v2 "ZooKeeper read" APIs have been tweaked slightly to be more intuitive. "get node data" is now available
at `GET /api/cluster/zookeeper/data/<path>`, and "list node children" is now available at `GET /api/cluster/zookeeper/children/<path>`
(Jason Gerlowski, Joshua Ouma)
- SOLR-16393: The path of the v2 "list alias" API has been tweaked slightly to be more intuitive, and is now available at
`GET /api/aliases`. It is also now possible to request information about a specific alias at `GET /api/aliases/<aliasName>`.
(Alex Deparvu via Jason Gerlowski)
- SOLR-16397: /mlt now has a v2 API available at `GET /api/collections/collName/mlt`
(Ameer Albahem via Jason Gerlowski)
- Optimizations (7)
- SOLR-16515: Remove synchronized access to cachedOrdMaps in SlowCompositeReaderWrapper
(Dennis Berger, Torsten Bøgh Köster, Marco Petris)
- SOLR-16555: SolrIndexSearcher - FilterCache intersections/andNot should not clone bitsets repeatedly
(Kevin Risden, David Smiley)
- SOLR-15732: queries to missing collection are slow
(noble)
- SOLR-15616: Allow thread metrics to be cached
(Ishan Chattopadhyaya, ab)
- SOLR-16615: Jersey 'ApplicationHandlers' are now shared by compatible cores where possible
(Jason Gerlowski, Houston Putman)
- SOLR-16487: Replication pooling is optimized based on hard and soft commit settings for a given collection
(Justin Sweeney)
- SOLR-16689: Avoiding commits on leader when recovering a non-leader replica and avoiding incorrectly replicating an empty core
(Justin Sweeney)
- Bug Fixes (33)
- SOLR-16436: Fix "false positive" suggestions from DirectSolrSpellChecker when using maxQueryFrequency > 1 in
multi-shard collections
(hossman)
- SOLR-15854: Upgrade semver library to fix versioning constraint bugs in the PackageManager.
(Houston Putman, Kevin Risden)
- SOLR-16160: UpdateXmlMessages duplicate data when data is removed and then added in the same message
(Alex Deparvu via Kevin Risden)
- SOLR-16483: Fix IndexOutOfBounds in RecursiveNumericEvaluator
(Bendegúz Ács via Kevin Risden)
- SOLR-16363: DirectUpdateHandler2 should not throw UnknownFormatConversionException
(Kevin Risden)
- SOLR-16293: Luke request fails for document with a binary field
(Kevin Risden)
- SOLR-16477: Collection RENAME api creates broken alias
(Alex Deparvu via Kevin Risden)
- SOLR-10918: Fix IntPointField hashing for HLL
(Houston Putman)
- SOLR-16274: HEAD request for managed resource returns 500 Server Error
(Kevin Risden)
- SOLR-16528: Jaegertracer module must include okhttp3 dependency
(janhoy)
- SOLR-16420: Default for cloud mode was fixed to `{!mlt mindf=5}` to comply with Reference Guide
(Mikhail Khludnev)
- SOLR-16473: Fix race condition in shard split when a sub-shard is put in recovery state.
(Andy Vuong via Bruno Roustant)
- SOLR-10458: Fix followRedirect property on HttpSolrClient not set when using Builder pattern.
(Eric Pugh)
- SOLR-16605: CPU percent calculation incorrect in admin UI cloud->nodes tab
(Shawn Heisey)
- SOLR-16611: NullPointerException occus when there are no segments in `{!collapse hint=top_fc}`
(Minami Takuya via Mikhail Khludnev)
- SOLR-16613: CryptoKeys should handle RSA padding for OpenJ9
(Kevin Risden)
- SOLR-16621: Admin UI fails to grant user permissions that have wildcard role
(janhoy)
- SOLR-16631: solr.allowUrls (former solr.shardsWhitelist) should treat hostnames in case insensitive way.
(Paul Blanchaert via Eric Pugh)
- SOLR-16647: Fix circuit breaker examples in solrconfig.xml
(Colvin Cowie via Kevin Risden)
- SOLR-16648: NullPointerException when excluding facets in More Like This Handler
(Mikhail Khludnev)
- SOLR-16628: Ensure that InputStreams are closed after Xml parsing
(Michael Gibney, David Smiley, Kevin Risden)
- SOLR-16639: Fix jq parse error of solr-exporter metrics node_thread_pool_completed_total
(Naoto Minami)
- SOLR-16658: List of permissions returned to Admin UI is not complete
(janhoy)
- SOLR-9698: Fix start/stop wait time and RMI_PORT on Windows
(Colvin Cowie)
- SOLR-16670: Fix directory/file check in S3Repository
(Houston Putman, Hakan Özler)
- SOLR-16668: Use default to Java SSL for Http2SolrClient when none is provided
(Houston Putman)
- SOLR-16669: Http2SolrClient now defaults checkPeerName to True, as the documentation specified
(Houston Putman)
- SOLR-16679: Fix solr.jetty.ssl.verifyClientHostName logging
(Kevin Risden)
- SOLR-16682: MoreLikeThis Component fails with SyntaxError: Cannot parse if document terms contains symbols from query parser syntax
(Mikhail Khludnev)
- SOLR-16656: rid parameter missing from query logs
(Alex Deparvu)
- SOLR-16676: Logs: Http2SolrClient.async() lacked MDC
(Alex Deparvu)
- SOLR-16686: Using bin/solr to copy a file from ZK to local fails if the local filename does not have a path.
(Shawn Heisey)
- SOLR-16653 : sub-shard remains inactive after split for NRT+PULL collection
(Hitesh Khamesra, noble)
- Build (3)
- SOLR-16476: Remove commons-text dependency from solr-core
(Kevin Risden)
- SOLR-16624: Remove Gradle Groovy version override
(Kevin Risden)
- SOLR-16641: Generate gradle.properties from gradlew, if absent
(Colvin Cowie)
- Dependency Upgrades (38)
- SOLR-15955: Upgrade to Jetty 10.x
(Mark Miller, Kevin Risden)
- PR#1052: Upgrade forbiddenapis to 3.4
(Uwe Schindler)
- SOLR-16432, SOLR-16508: Upgrade the gradle wrapper to 7.6
(Shawn Heisey, Kevin Risden)
- SOLR-16442: Upgrade to Lucene 9.4.2
(Christine Poerschke)
- SOLR-16578: Upgrade to errorprone 2.18.0
(Kevin Risden)
- SOLR-16625: Upgrade OWASP dependency check to 8.0.1
(Kevin Risden)
- SOLR-16545: Upgrade Carrot2 to 4.5.0
(Dawid Weiss)
- SOLR-16481: Upgrade Bats to 1.8.0
(Kevin Risden)
- SOLR-16467, PR#1426: Upgrade semver4j to v4, fixes failures with some locales
(Kevin Risden)
- SOLR-16529: Upgrade jaeger-client to 1.8.1
(janhoy)
- SOLR-16579, PR#1379: Upgrade Jackson to 2.14.2
(Kevin Risden)
- SOLR-16562, PR#1433: Upgrade to Caffeine 3.1.5
(Kevin Risden)
- SOLR-16626, PR#1392: Upgrade to Netty v4.1.89.Final
(Kevin Risden)
- SOLR-16627, PR#1399, PR#1401: Upgrade google-cloud-bom to v0.190.0, re2j to 1.7, and grpc to 1.53.0
(Kevin Risden)
- PR#1425: Update dependency org.jctools:jctools-core to v4
(solrbot)
- PR#1421: Update org.apache.logging.log4j:* to v2.20.0
(solrbot)
- PR#1420: Update org.apache.calcite:* to v1.33.0 and org.apache.calcite.avatica:* to v1.23.0
(solrbot)
- PR#1419: Update io.prometheus:* to v0.16.0
(solrbot)
- PR#1417: Update dependency org.openjdk.jmh:jmh-core to v1.36
(solrbot)
- PR#1416: Update dependency org.bitbucket.b_c:jose4j to v0.9.3
(solrbot)
- PR#1415: Update dependency org.apache.commons:commons-compress to v1.22
(solrbot)
- PR#1406: Update dependency net.sourceforge.argparse4j:argparse4j to v0.9.0
(solrbot)
- PR#1404: Update dependency joda-time:joda-time to v2.12.2
(solrbot)
- PR#1403: Update dependency commons-cli:commons-cli to v1.5.0
(solrbot)
- PR#1402: Update dependency com.tdunning:t-digest to v3.3
(solrbot)
- PR#1398: Update dependency com.fasterxml.woodstox:woodstox-core to v6.5.0
(solrbot)
- PR#1397: Update dependency biz.aQute.bnd:biz.aQute.bnd.annotation to v6.4.0
(solrbot)
- PR#1396: Update org.slf4j:* to v2.0.6
(solrbot)
- PR#1395: Update org.apache.zookeeper:* to v3.8.1
(solrbot)
- PR#1394: Update org.apache.tika:* to v1.28.5
(solrbot)
- PR#1393: Update org.apache.httpcomponents. httpclient v4.5.14, httpcore v4.4.16, httpmime v4.5.14
(solrbot)
- PR#1391: Update io.dropwizard.metrics:* to v4.2.17
(solrbot)
- PR#1390: Update dependency org.xerial.snappy:snappy-java to v1.1.9.1
(solrbot)
- PR#1388: Update dependency org.immutables:value-annotations to v2.9.3
(solrbot)
- PR#1387: Update dependency net.thisptr:jackson-jq to v0.0.13
(solrbot)
- PR#1383: Update dependency io.swagger.core.v3:swagger-annotations to v2.2.8
(solrbot)
- PR#1382: Update dependency com.lmax:disruptor to v3.4.4
(solrbot)
- PR#1381: Update dependency com.github.spotbugs:spotbugs-annotations to v4.7.3
(solrbot)
- Other Changes (22)
- SOLR-16150: Embedded ZK Server used for zkRun should advertise itself on specific bound interfaces instead of
generically on localhost.
(Mike Drob)
- SOLR-15718: Remove backcompat feature solr.useUnsafeOverseerResponse
(janhoy)
- SOLR-16541: Eliminate the 'numShards' system property
(noble)
- SOLR-15861: ConcurrentUpdateSolrClient should work with ManagedExecutorService
(Sammy Chu, Kevin Risden)
- SOLR-16575: splitshard should honour createNodeSet
(noble)
- SOLR-16569: Add java system property to overseer queue size
(Nick Ginther via noble)
- SOLR-16574: Demonstrate Dense Vectors and KNN as part of the Films example
(Gabriel Magno via Eric Pugh)
- SOLR-16534: Jaegertracer-Configurator is now deprecated. From v10.0 we'll only support OpenTelemetry
(janhoy)
- SOLR-14336: Warning about potentially old Solr version in Admin UI dashboard
(janhoy)
- SOLR-13243: Correct the initial capacity of the ZK operations to run in ShardLeaderElectionContextBase#runLeaderProcess
(Haythem Khiri)
- SOLR-16544: Improve documentation on how to contribute to Solr
(Justin Sweeney)
- SOLR-16591: The "Transient Cores" feature is now deprecated. In solr.xml "<transientCoreCacheFactory>"
no longer works; it wasn't documented either. Some internals were changed as well to simplify
the standard case of having no transient cores.
(David Smiley)
- SOLR-16629: Replace Solr thread safety annotations with JCIP.
(David Smiley)
- SOLR-16632: Add core name to periodic delete related log messages
(Bence Szabo via Andras Salamon)
- SOLR-15959: Deprecate loading of solr.xml from Zookeeper
(janhoy)
- SOLR-16573: Introduce EmbeddedSolrServerTestRule, a JUnit 4 TestRule for Solr testing
(Joshua Ouma, David Smiley)
- SOLR-16660: Deprecate analytics component
(janhoy)
- SOLR-15733: Separate out a solrj-streaming module
(Joel Bernstein, Kevin Risden)
- SOLR-15928: Dim 'Add Collection' and 'Create Alias' buttons when user lacks proper permissions
(janhoy)
- SOLR-16672: Simplify SolrCLI Healthcheck command's logic for obtaining a target collection
(Eric Pugh)
- SOLR-16684: Keep solr's opennlp-tools version in sync with Lucene
(janhoy)
- SOLR-16681: Throw exception when attempting to replace uniqueKey via fl in distributed request,
e.g. fl=old_id:id,id:new_id
(Mikhail Khludnev)
- Bug Fixes (4)
- SOLR-16589: Large fields with large=true can be truncated when using unicode values
(Kevin Risden)
- SOLR-16585: Fixed NPE when paginating MatchAllDocs with non-zero start offset, like q=*:*&start=10
(Michael Gibney)
- SOLR-16165: Rare Deadlock in SlotAcc initialization
(Justin Sweeney, noble)
- SOLR-16622: Replicas don't come up active after node restart
(noble, Ishan Chattopadhyaya)
- Other Changes (6)
- SOLR-16443: Upgrade Jackson bom to 2.13.4.20221013
(Kevin Risden)
- SOLR-16568: Upgrade woodstox-core to 6.4.0
(Kira Traynor)
- SOLR-16598: Upgrade Protobuf to 3.21.12
(David Smiley)
- SOLR-16567: Fixed problem with filtering and KNN search, especially when using post-filters
(Alessandro Benedetti)
- SOLR-16588: Fixed problem with default knn algorithm
(Elia Porciani via Alessandro Benedetti)
- SOLR-16480: ConfigSets now have an overridable allow-list for filetypes.
(Houston Putman)
- New Features (10)
- SOLR-16246: Introduced pre-filtering in KnnQParser
(Elia Porciani via Alessandro Benedetti).
- SOLR-16111: Add hl.queryFieldPattern support, an advanced alternative to the hl.requireFieldMatch boolean flag.
(Christine Poerschke, David Smiley)
- SOLR-15921: Load jars in <solr-install-dir>/lib/ by default
(janhoy)
- SOLR-16005: Support add/update/delete field types in Admin UI Schema page.
(Ahmet Can Kepenek, Eric Pugh)
- SOLR-15853: Admin UI support for managing Paramsets and using in Queries.
(Betul Ince, Eric Pugh)
- SOLR-16096: Support createNodeSet parameter when creating collections in Admin UI.
(Eric Pugh)
- SOLR-14319: Add ability to specify replica types when creating collections in Admin UI.
(Richard Goodman, Eric Pugh)
- SOLR-16282: CoreAdminHandler supports custom actions via solr.xml configuration.
(Artem Abeleshev, Christine Poerschke)
- SOLR-15007: Add ability to roll up core level metrics to be node level metrics for a RequestHandler via configuration.
(Justin Sweeney, David Smiley)
- SOLR-15715: Dedicated query coordinator nodes in the solr cluster
(noble, Hitesh Khamesra, Ishan Chattopadhyaya)
- Improvements (17)
- SOLR-15986: CommitUpdateCommand and SplitIndexCommand can write user commit metadata.
(Bruno Roustant)
- SOLR-15652: Add Slack channel for community support to Admin UI footer, update links.
(Eric Pugh)
- SOLR-16002: Avoid redundant `FilterQuery` caching, e.g. via `filter($query)` syntax.
(Michael Gibney, David Smiley)
- SOLR-15045: `DistributedZkUpdateProcessor` now issues commits to local shards and remote shards in parallel,
halving the latency of synchronous commits
(Michael Gibney)
- SOLR-16181: Initialize the LogWatcher earlier in CoreContainer#load()
(janhoy)
- SOLR-16192: Add ZK credentials injectors support
(Lamine Idjeraoui with review by Anshum Gupta, Jason Gerlowski, Mike Drob and Houston Putman)
- SOLR-16225: Upgrade dependencies (Carrot2, HPPC)
(Dawid Weiss)
- SOLR-16257: Improve ZkStateReader to avoid race condition between collectionWatches and watchedCollectionStates
(Patson Luk, Houston Putman, Mike Drob)
- SOLR-16323: The Docker image now uses the Solr User ID instead of the User Name, helps with non-root checks
(Houston Putman)
- SOLR-16340: Provide a cluster_id label for all metrics exposed by the Solr Prometheus Exporter, making it possible to
filter Grafana board by cluster
(janhoy, Houston Putman)
- SOLR-16337: implement Zk metrics
(noble)
- SOLR-16353: Make SplitShardCmd#checkDiskSpace disableable through a system property.
(Haythem Khiri)
- SOLR-16362: Logs: Truncate field values in logs if a doc fails to index.
(Nazerke Seidan, David Smiley)
- SOLR-16230: JWT nested roles support
(Marco Descher, janhoy)
- SOLR-16361: mod() is now accurate for all integers, floats, doubles and longs upto 2^52
(Dan Rosher via Eric Pugh)
- SOLR-16404: Prometheus Exporter: Use HTTP2 Solr clients.
(Houston Putman)
- SOLR-16429: The s3-repository module now supports AWS STS, and therefore Web Identity Tokens, by default.
(Josh Souza via Houston Putman)
- Optimizations (8)
- SOLR-16120: Optimise hl.fl expansion.
(Christine Poerschke, David Smiley, Mike Drob)
- SOLR-14765: Optimize DocList creation for sort-irrelevant cases. This issue also reworks the building and caching of liveDocs
in SolrIndexSearcher, and refines/clarifies the effect of `useFilterForSortedQuery`
(Michael Gibney, Mike Drob, David Smiley)
- SOLR-16146: Avoid loading all collections during node startup
(noble)
- SOLR-16266: Eliminate unneccessary byte[] copy in Http2SolrClient
(hossman)
- SOLR-9359: Enable warming queries to be managed using Config API
(Andy Webb, Eric Pugh)
- SOLR-16336: avoid fetching solrconfig.xml & schema.xml for already cached schema and config
(noble)
- SOLR-16328: intern() strings in DocCollection to reduce memory footprint
(noble)
- SOLR-16445: Leader message should only be sent when there is more than one (NRT+Tlog) replica
(Hitesh Khamesra, noble via Ishan Chattopadhyaya)
- Bug Fixes (45)
- SOLR-13219: Fix NPE in FieldLengthFeature with non-stored/missing fields
(Nick Veenhof, Tomasz Elendt)
- SOLR-15918: Skip repetitive parent znode creation on config set upload
(Mike Drob)
- SOLR-15964: Transient cores: don't evict a core when it's still being used.
(David Smiley)
- SOLR-15116: Support a HEAD request for managed resources.
(Eric Pugh)
- SOLR-16089: Fix empty columns in Cloud Node UI Screen when replica is in down state.
(hossman)
- SOLR-16112: DefaultSolrHighlighter.doHighlighting to Query#rewrite multiple times if necessary.
(Christine Poerschke)
- SOLR-13044: Fix NPE during core close racing with index replication check.
(David Smiley, hossman)
- SOLR-16145: Fix very rare NPE in SolrCloud rejoinOverseerElection.
(David Smiley)
- SOLR-16168: Spellcheck NPE if invalid dictionary name is provided
(Kevin Risden)
- SOLR-15830: Concurrent core reloads mess up commits when using Schema API
(Bence Szabo via Andras Salamon)
- SOLR-16169: IndexBasedSpellChecker with empty spellcheck.q results in NegativeArraySizeException
(Kevin Risden)
- SOLR-16110: Using Schema/Config API breaks the File-Upload of Config Set File
(Steffen Moldenhauer, Kevin Risden)
- SOLR-16218: failOnVersionConflicts option not working for in-place updates
(Lamine Idjeraoui)
- SOLR-16259: Missing newline at the end of solr.in.sh breaks the installer.
(John Gately via Shawn Heisey)
- SOLR-16241: Fix JettyConfig.builder(JettyConfig) to correctly copy all attributes
(hossman)
- SOLR-16142: Fix Admin UI's spatial parameter generation.
(Arsal Jalib, Christine Poerschke)
- SOLR-9661: Fix explanation of select() streaming expression that uses replace() operation
(Ahmet Can Kepenek via Eric Pugh)
- SOLR-16316: Fix debugQuery clicking on the Query UI does not show the debug.explain.structured option.
(Shiming Li via Eric Pugh)
- SOLR-16399: ExportWriter fails with max values for fields
(Kevin Risden)
- SOLR-16357: CaffeineCache ramBytes incorrect accounting on updates and deletes
(Alex Deparvu)
- SOLR-16343: Handle MDC snapshot being null
(Dee Moore via Kevin Risden)
- SOLR-7883: MoreLikeThisHandler to support facets
(Mikhail Khludnev)
- SOLR-16406: Take SOLR_MODULES, configured by solr.in.sh, into account.
(Thomas Wöckinger via janhoy)
- SOLR-16344: PlacementPlugin throws NPE for PRS collections
(noble)
- SOLR-16417: NPE if facet query hits timeout or exception
(Kevin Risden)
- SOLR-16414: Race condition in PRS state updates
(noble, Justin Sweeney, Patson Luk, Hitesh Khamesra, Ishan Chattopadhyaya)
- SOLR-16418: Introduce SolrResponseUtil to handle NPE during query timeout or exception when parsing SolrResponse
(Kevin Risden)
- SOLR-16219: ICUCollationField protected field IllegalAccessException from different classloader
(Michael Gibney)
- SOLR-16426: LBHttp2SolrClient (and therefore CloudHttp2SolrClient) now respect requestWriter, responseParser and queryParam changes
(Houston Putman)
- SOLR-16232: Fix EnvVar usage in bin/solr
(Houston Putman)
- SOLR-16439: Cloud Node does not have Core, the admin UI cannot be displayed properly.
(Shiming Li)
- SOLR-16229: Http2SolrClient aborts requests on exception
(Daniel Rabus, Tomás Fernández Löbbe)
- SOLR-16457: solr.data.home should not be set to empty string in bin/solr
(Kevin Risden)
- SOLR-16433: Security Manager prevents Solr SQL from working
(Kevin Risden)
- SOLR-16460: ClusterState.copyWith is inconsistent
(noble)
- SOLR-16440: RefreshCollectionMessage in PRS should update Overseer's ZkStateWriter
(noble, Ishan Chattopadhyaya)
- SOLR-16453: Overseer doesn't handle PRS and non-PRS messages properly
(Justin Sweeney, Hitesh Khamesra)
- SOLR-16463: Workaround for serious crash on JDK17+ due to JIT on caffeinecache
(janhoy, Uwe Schindler)
- SOLR-16478 :removing a replica should delete the PRS entry from the hostnode and not from overseer
(Patson Luk via noble)
- SOLR-16485: Fix NPE in ShardHandlerFactory when running in Standalone mode
(Houston Putman)
- SOLR-16412: Race condition could trigger error on concurrent SizeLimitedDistributedMap cleanup
(Patson Luk via noble)
- SOLR-16416: OverseerPrioritizer now runs after all handlers are registered, and retries on failures.
(Houston Putman)
- SOLR-14679: Fix continuously growing TLOGs on TLOG replicas
(Viktor Molnár via Houston Putman)
- SOLR-16502: Multiple CopyField should not limit to first maxChars
(Fredrik Rodland, Kevin Risden)
- SOLR-16527: RuleBasedAuthorizationPluginBase NPE
(Alex Deparvu)
- Other Changes (34)
- SOLR-16351: Upgrade Carrot2 to 4.4.3, upgrade randomizedtesting to 2.8.0.
(Dawid Weiss)
- SOLR-16245: Make DenseVectorField codec agnostic
(Elia Porciani via Alessandro Benedetti)
- SOLR-15897: Remove <jmx/> from all unit test solrconfig.xml files.
(Eric Pugh)
- SOLR-15776: Admin UI is now aware of logged-in user's permissions and can adapt accordingly
(janhoy)
- SOLR-15923: Cue users in 404 message that Solr is running and the url is incorrect.
(Eric Pugh, Mike Drob, Shawn Heisey)
- SOLR-9529: Consistency in how dynamic fields are defined in sample schema.xml files
(Eric Pugh, Trey Grainger)
- SOLR-15886: Remove deprecated showItems configuration value from solrconfig.xml files
(Andy Lester via Eric Pugh)
- SOLR-16114: SolrZooKeeper has been removed in favor of using ZooKeeper directly.
(Mike Drob)
- SOLR-16154: Event listeners submit through core container executor service instead of separate thread
(Mike Drob, Kevin Risden)
- SOLR-16190: Http2SolrClient should make sure to shutdown the executor
(Kevin Risden)
- SOLR-16191: Validate that installed ps utility supports -p flag, so that we do not inadvertantly stop the wrong process.
(Mike Drob, Michael Gibney)
- SOLR-16209: Rolling restart will no longer trigger as much PKI Plugin error logging.
(Mike Drob, Tomás Fernández Löbbe)
- SOLR-16213: Upgrade Jackson to 2.13.3
(janhoy)
- SOLR-16231: Fix ref-guide links across the codebase
(Houston Putman)
- SOLR-16304: No more @Slow annotation for tests!
(Mike Drob)
- SOLR-16331: Migrate remaining managed-schema files to be managed-schema.xml.
(Eric Pugh)
- SOLR-16332: Upgrade Jetty to 9.4.48.v20220622
(Chris Sabelstrom, janhoy)
- SOLR-16339: Refined "no servers hosting shard" SolrException wording.
(Christine Poerschke)
- SOLR-15342: SolrJ: Split ZooKeeper based dependencies off to a solr-solrj-zookeeper module /
dependency.
(David Smiley, Haythem Khiri)
- SOLR-16324: Upgrade commons-configuration2 to 2.8.0 and commons-text to 1.8
(Kevin Risden)
- SOLR-16296: XmlConfigFile and QueryElevationComponent now use SafeXMLParsing.
(Haythem Khiri)
- SOLR-16369: Avoid XPath in parsing elevate.xml
(Haythem Khiri)
- SOLR-16402: Bump google-cloud-bom to 0.178.0
(Kevin Risden)
- SOLR-16141: Upgrade Apache Tika to 1.28.4
(janhoy)
- SOLR-16424: Upgrade Netty to 4.1.82.Final
(Kevin Risden)
- SOLR-16421: Upgrade Apache Calcite to 1.32.0 and Avatica to 1.22.0
(Kevin Risden)
- SOLR-16422: Upgrade Apache Zookeeper to 3.8.0
(Kevin Risden)
- SOLR-16423: Upgrade Apache Hadoop to 3.3.4
(Kevin Risden)
- SOLR-16450: Proper handling on watcher registration failure in ZkStateReader#waitForState()
(Hitesh Khamesra via Ishan Chattopadhyaya)
- SOLR-16451: Don't fetch the PRS states while registering the collection watch
(Hitesh Khamesra via Ishan Chattopadhyaya)
- SOLR-16452: Do not update PRS states if local version is newer
(Hitesh Khamesra via noble)
- SOLR-16464: Upgrade commons-text to 1.10.0
(Kevin Risden)
- SOLR-16482: Add an internal envVar for passing options to bin/solr that will not be overriden by the user.
(Houston Putman)
- SOLR-16486: Pin the OS Variant for the default base image of the Dockerfile (to Ubuntu 20).
(Houston Putman, janhoy, Eva Müller)
- Build (9)
- SOLR-16204: Change Lucene dependency to Lucene 9.1.0
(Elia Porciani via Alessandro Benedetti)
- SOLR-16303: Change Lucene dependency to Lucene 9.3.0
(Mike Drob)
- SOLR-16053: Upgrade scriptDepVersions
(Kevin Risden)
- SOLR-16050: Upgrade to errorprone 2.11.0
(Kevin Risden)
- Upgrade forbiddenapis to 3.3
(Uwe Schindler)
- SOLR-16258: Gradle no longer uses "errorprone" by default
(Dawid Weiss, Uwe Schindler, hossman)
- SOLR-16264: Set versions for all Antora build tools
(Houston Putman)
- SOLR-16299: Ref-guide examples are no longer symlinked from SolrJ, they live in the ref-guide module
(Houston Putman)
- SOLR-16298: Clean the javadocs directory before building
(Houston Putman, Mike Drob, Uwe Schindler)
- New Features (18)
- SOLR-14440: Introduce new Certificate Authentication Plugin to load Principal from certificate subject.
(Mike Drob)
- SOLR-13528 Rate Limiting in Solr
(Atri Sharma, Mike Drob)
- SOLR-14749: Provide a clean API for cluster-level event processing.
Improve support for arbitrary container-level plugins. Add ClusterSingleton
support for plugins that require only one active instance in the cluster.
(ab, noble)
- SOLR-14613, SOLR-15019: Autoscaling replacement using placement plugins
(ilan, ab, noble)
- SOLR-15055: Re-implement 'withCollection'. This also adds the placement plugin support
for rejecting replica / collection deletions that would violate placement constraints.
(ab, ilan)
- SOLR-15130: Support for per-collection replica placement node sets, a.k.a "node type"
placements.
(ab, ilan)
- SOLR-8138: Simple UI for issuing SQL queries
(Michael Suzuki via Eric Pugh)
- SOLR-14787: Payload check query parser now supports inequalities.
(Kevin Watters, Gus Heck)
- SOLR-15164: Implement Task Management Interface
(Atri Sharma, with extensive review and perf testing by Anshum Gupta, Mike Drob and Houston Putman)
- SOLR-15300: Report collection and shard "health" state in CLUSTERSTATUS response.
(ab, janhoy)
- SOLR-15423: JWTAuthPlugin now supports separate config for what SSL certs to trust when talking to IdPs
(janhoy)
- SOLR-15694: Node roles framework, allowing restriction of certain nodes to certain tasks
(Ishan Chattopadhyaya, noble)
- SOLR-15197: Support temporal graph queries with DAY and WEEKDAY windows
(Joel Bernstein)
- SOLR-15914: Official Solr modules (contribs) can now easily be added to shared class path by environment variable
'SOLR_MODULES' or system property 'solr.modules'. E.g: SOLR_MODULES=extracting,langid
(janhoy, David Smiley, Houston Putman)
- SOLR-15880: Introduce support for k nearest neighbors search
(Alessandro Benedetti, Elia Porciani)
- SOLR-14660: Move HDFS support to a new HDFS module
(Istvan Farkas, Kevin Risden)
- SOLR-13989: Move Hadoop Authentication support to a new hadoop-auth module
(Kevin Risden)
- SOLR-15904: Move SQL support to new sql module
(Kevin Risden)
- Improvements (69)
- LUCENE-8984: MoreLikeThis MLT is biased for uncommon fields
(Andy Hind via Anshum Gupta)
- SOLR-14223: PKI Auth can bootstrap from existing key files instead of creating new keys on startup
(Mike Drob)
- SOLR-15153: Admin UI: Collection selector drop down does not sort collections
(Edward Ribeiro)
- SOLR-11725: Use corrected sample formula for computing stdDev and variance in JSON aggregations
(hossman, Munendra S N, yonik)
- SOLR-14387: SolrClient.getById() will escape comma separator within ids
(Markus Schuch via Mike Drob)
- SOLR-15327: Fix typos in the code base
(Edward Ribeiro via Eric Pugh)
- SOLR-10814: Add short-name feature to RuleBasedAuthz plugin
(Mike Drob, Hrishikesh Gadre)
- SOLR-7683: Introduce support to identify Solr internal request types
(Atri Sharma, Hrishikesh Gadre)
- SOLR-14799: JWT authentication plugin only requires "sub" claim when principalClaim=sub.
(Erik Hatcher)
- SOLR-14878: Report solr.xml's coreRootDirectory property via System Settings API, when set
(Alexandre Rafalovitch)
- SOLR-14880: Support coreRootDirectory setting when creating new cores from command line in standalone mode
(Alexandre Rafalovitch)
- SOLR-14926, SOLR-14926, SOLR-13506: Modernize and clean up search results clustering module. This issue upgrades
the clustering module to the new Carrot2 4.x line, dropping several CVE-prone dependencies along the way.
The parameters and configuration of the module extensions have changed. The documentation in Solr ref guide
has been rewritten from scratch to be up to date. Clustering code has been rewritten from scratch to work
properly regardless of the mode (standalone, distributed). The API has been stripped of ancient, unused, interfaces
and simplified.
(Dawid Weiss)
- SOLR-14972: Prometheus: Change default port of prometheus exporter to 8989 because it clashed with default
embedded zookeeper port
(janhoy)
- SOLR-15011: /admin/logging handler will now propagate setLevel (log threshold) to all nodes
when told to. The admin UI now tells it to.
(Nazerke Seidan, David Smiley)
- SOLR-15100: Make the ConfigSetService pluggable/configurable via <string name="configSetService" /> in solr.xml
(baisui)
- SOLR-15185: Various optimizations to the {!hash} QParser, typically used by the parallel()
streaming expression. The hash algorithm changed.
(David Smiley)
- SOLR-2852: SolrJ: remove Woodstox dependency. It was never truly required there.
Software doing lots of XML processing can choose to add it or alternatives if they wish.
(David Smiley)
- SOLR-15161: Don't encourage users to hack JSON response mimetype by documenting in examples how to
specify wt=json use mimetype of text/plain.
(Eric Pugh)
- SOLR-15276: V2 API call to look up async request status restful style of "/cluster/command-status/1000" instead
of "/cluster/command-status?requestid=1000".
(Eric Pugh)
- SOLR-15274: The QueryElevationComponent now supports loading elevation file changes on commits.
This doesn't work in SolrCloud (but may someday). QEC no longer supports a config file in
the data dir.
(David Smiley)
- SOLR-14185: Added DocSet.iterator(LeafReaderContext) and some related changes that may add
a minor performance boost to some cases (e.g. interval facets). Reduced need for Filter.java.
(Michael Gibney, David Smiley)
- SOLR-15329: Improve HDFS Directory size calculation
(Andras Salamon via Mike Drob)
- SOLR-15340: Rename shardsWhitelist and extract AllowListUrlChecker to use it more broadly.
(Bruno Roustant)
- SOLR-15414: Use ConfigSet API instead of Zookeeper data node to list out configsets available in Solr Admin UI.
(Nazerke Seidan via Eric Pugh)
- SOLR-15421: ConfigSet API also checks for solrconfig.xml when checking the existence of a configset
(Andras Salamon via David Smiley)
- SOLR-15392: Distributed Tracing request span operation names are now composed of a command/verb
and a templated path. The collection or core name is now in the db.instance tag.
(David Smiley)
- SOLR-15453: Update the content security policy in Jetty to allow image requests from local host and prevent security
errors on the client side.
(MarcusSorealheis via Houston Putman)
- SOLR-15362: Let core and collection dropdowns in Admin UI float wide to see entire core or collection name.
(Matthias Krepp, Eric Pugh)
- SOLR-15044: When indexing nested docs via JSON, it is no longer necessary to provide child doc IDs.
This was already working for XML & "javabin"/SolrJ. Previously, omitting the ID would be confused
for a partial/atomic update.
(David Smiley)
- SOLR-10887: Migrate "managed-schema" file naming to "managed-schema.xml" file name, with a fallback to the legacy
"managed-schema".
(Eric Pugh, David Smiley)
- SOLR-15630: Logging MDC values no longer include a hardcoded prefix, allowing custom logging configurations access to
the plain values. The default log4j2.xml PatternLayout has been updated to ensure the values are formatted with the
existing prefixes.
(hossman)
- SOLR-15650: Choosing lucene defType in Solr Admin now is passed explicitly through UI, not relying on default
solrconfig.xml behavior.
(Eric Pugh)
- SOLR-12848: SolrJ and the server can now recognize some standard Java system properties like
for an HTTP proxy. This is only for the Apache HttpClient based SolrJ communication, not Jetty.
(Shawn Heisey, David Smiley)
- SOLR-15301: Eliminate repetitive index size calculation for Solr metrics
(Andras Salamon)
- SOLR-15617: Add kerberos.name.rules.mechanism support to KerberosPlugin
(Kevin Risden)
- SOLR-15278: Add V2 equivalent to allow deleting async collection list of statuses.
(Eric Pugh)
- SOLR-11623: Every request handler in Solr now implements PermissionNameProvider to explicitly decide on what security
permissions are required to access the handler
(janhoy, Hrishikesh Gadre, David Smiley)
- SOLR-15705: A delete-by-id command is forwarded to all shards when using the CompositeId router with a router field
and the route is missing from the command.
(Michael Kosten via Christine Poerschke, David Smiley, Eric Pugh)
- SOLR-15790: SearchHandler now includes the `rid` value in the Logging MDC for the duration of the request, allowing
custom logging configurations to include it.
(hossman)
- SOLR-15785: Custom node request handlers/endpoints that weren't in packages can now be located in JARs in solr-home/lib,
not just WEB-INF/lib.
(Nazerke Seidan, David Smiley)
- SOLR-15376: Accept "Long" values for CollectionAdminRequest.CreateTimeRoutedAlias.setMaxFutureMs
to support durations greater than ~ 25 days.
(Nahian-Al Hasan, Gus Heck, Christine Poerschke)
- SOLR-15427: Nested docs: [child limit=...] now defaults to -1 which is interpreted as unlimited.
(David Smiley)
- SOLR-15786: Add the "films" example to SolrCLI via -e films parameter.
(Eric Pugh)
- SOLR-15834: Films example readme needs updating, including useParams support for multiple algorithms.
(Eric Pugh)
- SOLR-15824: Improved Query Screen handling of raw query parameters.
(Betul Ince via Tim Potter, Eric Pugh)
- SOLR-15803: Compute concurrent replica assignment requests together, using the shared context to better distribute
replicas.
(Houston Putman)
- SOLR-15213: Atomic updates: "add" now uses add-or-replace logic for child documents. They can
also themselves be atomic updates.
(James Ashbourne, Endika Posadas via David Smiley)
- SOLR-10321: Highlighting (hl.method=unified): When there are no highlights for a field, don't
return the field in the response at all.
(David Smiley)
- SOLR-15890: Add a limit to the Admin SQL panel if one is not included in the stmt
(Joel Bernstein)
- SOLR-15887: Remove <jmx/> from shipped solrconfig.xml.
(Eric Pugh)
- SOLR-14608: Faster sorting for the /export handler
(Joel Bernstein)
- SOLR-15209: The old LegacyAssignStrategy has been refactored into the SimplePlacementPlugin. This is still the default
placement policy for Solr.
(Houston Putman, Ilan Ginzburg)
- SOLR-15257: Replace DocSet.getTopFilter with DocSet.makeQuery. Create DocSetQuery which is a Query
and DocSetProducer.
(Collins Abanda, Mike Drob, Tim Potter, David Smiley, Michael Gibney)
- SOLR-12336: Remove Filter, SolrFilter and SolrConstantScoreQuery. Filter no longer has a need to exist due to multiple JIRA issues,
implemented by a number of issues. With Filter going away, there is no longer need for SolrConstantScoreQuery.
(Collins Abanda, Mike Drob, Tim Potter, David Smiley, Michael Gibney)
- SOLR-14916: Add split parameter to timeseries Streaming Expression
(Joel Bernstein)
- SOLR-14686: Logs: Removed the "[corename]" prefix of some SolrCore logs that has become redundant
with MDC.
(David Smiley)
- SOLR-15884: Backup responses now use a map to return information instead of a list
(Houston Putman, Christine Poerschke)
- SOLR-9376: Raw value DocTransformers (`[xml]`, `[json]`, `[geo w=GeoJSON]`) now work
in a distributed/SolrCloud context
(Michael Gibney)
- SOLR-15755: Parse collection, shard, core, replica info from newSearcher events in postlogs tool
(Joel Bernstein)
- SOLR-9575: Solr no longer requires a solr.xml to be present in $SOLR_HOME. If one is not found, Solr will load
the one in $SOLR_TIP/server/solr/solr.xml. Solr also does not require a zoo.cfg in $SOLR_HOME if started with
embedded zookeeper.
(janhoy)
- SOLR-11905: New Admin UI Query screen input boxes for JSON query and facet DSL
(janhoy)
- SOLR-15556: Migrate the Ref Guide to be built with Antora, enabling many new features
(Cassandra Targett, Houston Putman, Mike Drob)
- SOLR-15965: Use better signatures for the PKI Authentication plugin.
(Mike Drob)
- SOLR-15982: Add end time value to backup response, standardize backup response key names and date formats
(Artem Abeleshev, Christine Poerschke, Houston Putman)
- SOLR-14401: Metrics: Only SearchHandler and subclasses have "local" metrics now.
It's now tracked as if it's another handler with a "[shard]" suffix, e.g. "/select[shard]".
There are no longer ".distrib." named metrics; all metrics are assumed to be such except
"[shard]". The default Prometheus exporter config splits that component to a new label
named "internal". The sample Grafana dashboard now filters to include or exclude this.
(David Smiley)
- SOLR-16088: De-couple Http2SolrClient and ContentStreamBase from org.apache.http
(janhoy)
- SOLR-15286: A brand new follower in the legacy mode should wait to replicate index before reporting
healthy
(Pushkar Raste, Anshum Gupta)
- SOLR-15590: Start CoreContainer via ServletContextListener
(Gus Heck)
- SOLR-16131: Jetty threads' contextClassLoaders are set to the Solr core's resourceLoader's classLoader.
This allows for custom plugins and Solr modules to rely on 3rd part dependencies that use the thread contextClassLoader.
(Houston Putman)
- Build (15)
- LUCENE-9077 LUCENE-9433: Support Gradle build, remove Ant support from trunk
(Dawid Weiss, Erick Erickson, Uwe Schindler et.al.)
- LUCENE-9411: Fail compilation on warnings, 9x gradle-only (Erick Erickson, Dawid Weiss)
Deserves mention here as well as Lucene CHANGES.txt since it affects both.
- SOLR-13671: Remove check for bare "var" declarations in validate-source-patterns
(Erick Erickson, Alex Bulygin, janhoy)
- SOLR-14857: Add optional gradle property to run docker tests in parallel
(hossman, Houston Putman)
- SOLR-15793: Pin http_parser.rb to specific version to allow Solr Ref Guide on Jekyll to run
(Eric Pugh)
- SOLR-15867: Make the gradle build ready for 9.0 release
(janhoy, Houston Putman, Dawid Weiss)
- SOLR-15852: Update dev-tools/scripts for the 9.0 release, including releaseWizard
(janhoy)
- SOLR-15891: The Solr Docker image uses the default file permissions from the tarball artifact
(Houston Putman, janhoy)
- SOLR-15603: Add an option to activate Gradle build cache, build task cleanups
(Alexis Tual, Dawid Weiss)
- SOLR-15613: Enforce error-prone checks to catch all URLEqualsHashCode violations and use java.net.URI instead
(Collins Abanda)
- SOLR-15670: Introduce Gradle parameter to skip ref-guide tasks
(Houston Putman, Dawid Weiss)
- SOLR-15984: Ensure all used dependencies are declared
(Kevin Risden)
- SOLR-15987: Upgrade slf4j to 1.7.35 and remove ant-compat/force-versions.gradle
(Kevin Risden)
- SOLR-15992: Globally forbid and exclude known bad dependencies
(Kevin Risden)
- SOLR-12228: Move bin-test scripts to packaging project and create gradle task for running them.
(Mike Drob)
- Docker (12)
- SOLR-14789: Migrate docker image creation from docker-solr repo to solr/docker.
(Houston Putman, Martijn Koster, Tim Potter, David Smiley, janhoy, Mike Drob)
- SOLR-14790: Docker: Move Solr Docker image documentation to the ref guide.
(Houston Putman)
- SOLR-15322: Solr releases now contain everything needed to build runnable docker images
(Houston Putman, hossman)
- SOLR-15949: Docker: the official image now uses Java 17 provided by Eclipse Temurin. Formerly it was Java 11 from OpenJDK.
(janhoy, David Smiley)
- SOLR-15891: The Solr Docker image uses the default file permissions from the tarball artifact
(Houston Putman, janhoy)
- SOLR-14001: Docker: Removed /var/solr initialization from the Dockerfile; depend on init_var_solr.sh instead.
This leads to more consistent behavior no matter how /var/solr is mounted.
- init_var_solr.sh is now invoked by docker-entrypoint.sh; not in a bunch of other places.
- as before, you can set NO_INIT_VAR_SOLR=1 to short-circuit this.
- init_var_solr.sh no longer echo's anything. For verbosity, set VERBOSE=yes.
(David Smiley)
- SOLR-14957: Docker, Prometheus: Add Prometheus Exporter to docker PATH. Fix classpath issues.
(Houston Putman)
- SOLR-14949: Docker: Ability to customize the FROM image when building.
(Houston Putman)
- SOLR-15335: Docker: Solr has capability to build functionally-identical local and official Docker image.
(hossman, Houston Putman)
- Deprecation Removals (21)
- SOLR-15203: JWT Auth plugin: Remove deprecated parameter name jwkUrl in favour of jwksUrl for the JWK Url.
(Eric Pugh)
- SOLR-15608: Remove deprecated methods, classes and constructors from solrj clients
(janhoy)
- SOLR-14272: Remove autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay for 9.0 as it was
deprecated in 7.1
(Anshum Gupta)
- SOLR-14197: SolrResourceLoader: remove deprecated methods and do other improvements.
(David Smiley)
- SOLR-9909: The deprecated SolrjNamedThreadFactory has been removed. Use SolrNamedThreadFactory instead.
(Andras Salamon, shalin)
- SOLR-14783: Remove Data Import Handler (DIH), previously deprecated
(Alexandre Rafalovitch)
- SOLR-14035: Remove deprecated preferLocalShards=true support in favour of the shards.preference=replica.location:local
alternative.
(Alex Bulygin via Christine Poerschke)
- SOLR-14934: Remove redundant deprecated "solr.solr.home" logic
(hossman)
- SOLR-14034: Remove deprecated min_rf references
(Tim Dillon)
- SOLR-13893: Remove support to read BlobRepository's max jar size from deprecated `runtme.lib.size` system property
(Erick Erickson, Kesharee Nandan Vishwakarma, Munendra S N)
- SOLR-12720: Remove support for `autoReplicaFailoverWaitAfterExpiration`.
(marcussorealheis, shalin)
- SOLR-12823: Remove /clusterstate.json support, including support for collections created with stateFormat=1,
as well as support for Collection API MIGRATESTATEFORMAT action and support for the legacyCloud flag
(Ilan Ginzburg).
- SOLR-12847: Remove support for maxShardsPerNode.
(ab)
- SOLR-14244: Remove ReplicaInfo.
(ab)
- SOLR-14654: Remove plugin loading from .system collection
(noble)
- SOLR-14944: Remove the "spins" metrics - support for detection of spinning disks has been
removed in LUCENE-9576.
(ab)
- SOLR-15341: Remove indexHeapUsageBytes info from /admin/segments and /admin/luke because it's no
longer available in Lucene -- LUCENE-9387.
(janhoy, David Smiley)
- SOLR-15416: Remove metrics history collection (and MetricsHistoryHandler).
(ab)
- SOLR-13138: Remove deprecated LegacyBM25SimilarityFactory class.
(Christine Poerschke, janhoy)
- SOLR-15716: Remove deprecated SolrException.ignorePatterns and related code
(hossman)
- SOLR-12336: Remove Filter, SolrFilter and SolrConstantScoreQuery. Filter no longer has a need to exist due to multiple
JIRA issues, implemented by a number of issues. With Filter going away, there is no longer need for
SolrConstantScoreQuery.
(Collins Abanda, Mike Drob, Tim Potter, David Smiley, Michael Gibney)
- Other Changes (104)
- SOLR-14656: Autoscaling framework removed
(Ishan Chattopadhyaya, noble, Ilan Ginzburg)
- SOLR-14616: CDCR support removed
(Ishan Chattopadhyaya)
- LUCENE-9391: Upgrade HPPC to 0.8.2.
(Haoyu Zhai)
- SOLR-10288: Remove non-minified JavaScript from the webapp.
(Erik Hatcher, marcussorealheis)
- SOLR-13655: Upgrade Collections.unModifiableSet to Set.of and Set.copyOf
(Atri Sharma via Tomás Fernández Löbbe)
- SOLR-13797: SolrResourceLoader no longer caches bad results when asked for wrong type
(Mike Drob)
- LUCENE-9092: Upgrade Carrot2 to 3.16.2
(Dawid Weiss).
- LUCENE-9080: Upgrade ICU4j to 62.2 and make regenerate work
(Erick Erickson)
- SOLR-14271: Remove duplicate async id check meant for pre Solr 8 versions
(Anshum Gupta)
- SOLR-14272: Remove autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay for 9.0 as it was
deprecated in 7.1
(Anshum Gupta)
- SOLR-14258: DocList no longer extends DocSet.
(David Smiley)
- SOLR-14256: Remove HashDocSet; add DocSet.getBits() instead. DocSet is now strictly immutable and ascending order.
It's now locked-down to external extension; only 2 impls exist.
(David Smiley)
- SOLR-14012: Return long value for unique and hll aggregations irrespective of shard count
(Munendra S N, hossman)
- SOLR-14322: AbstractFullDistribZkTestBase.waitForRecoveriesToFinish now takes a timeout and time unit instead of
assuming that we are passed value in seconds.
(Mike Drob)
- SOLR-14420: AuthenticationPlugin.authenticate accepts HttpServletRequest instead of ServletRequest.
(Mike Drob)
- SOLR-14429: Convert .txt files to properly formatted .md files.
(Tomoko Uchida, Uwe Schindler)
- SOLR-14412: Automatically set urlScheme to https when running secure solr with embedded zookeeper. (Mike Drob)
Do not erroneously set solr.jetty.https.port system property when running in http mode
(Upendra Penegalapati)
- SOLR-14014: Introducing a system property that allows users to disable the Admin UI, which is enabled by default.
If you have security concerns or other reasons to disable the Admin UI, you can modify `SOLR_ADMIN_UI_DISABLED`
`solr.in.sh`/`solr.in.cmd` at start.
(marcussorealheis)
- SOLR-14702: All references to "master" and "slave" replaced with "leader" and "follower"
(MarcusSorealheis,
Erick Erickson, Tomás Fernández Löbbe)
- SOLR-14846: Backup/Restore classes no longer take Optional method parameters and will accept nulls instead.
(Mike Drob)
- LUCENE-9531: Consolidated CharStream and FastCharStream classes: these have been moved from each query parser
package to org.apache.lucene.queryparser.charstream
(Dawid Weiss).
- SOLR-9607: Remove /terms configuration from solrconfig.xml, as implicit definition is up-to-par now
(Alexandre Rafalovitch)
- SOLR-14036: Remove distrib=false from /terms handler's default parameters
(David Smiley, Munendra S N)
- SOLR-14829: Cleanup the documentation for Request Handlers and Search Components
(Alexandre Rafalovitch)
- SOLR-10370: ReplicationHandler should fetch index at fixed delay instead of fixed rate
(Cao Manh Dat)
- SOLR-14930: Removed rule based replica placement
(noble)
- SOLR-14912: Clean up solr-extraction module to produce solr-extraction-* jar
(instead of solr-cell-*).
(Dawid Weiss)
- SOLR-14915: Prometheus: Reduced dependencies from Solr server down to just SolrJ. Don't add WEB-INF/lib.
- Can run via gradle, "gradlew run"
- Has own log4j2.xml now
- Was missing some dependencies in lib/; now has all except SolrJ & logging.
(David Smiley, Houston Putman)
- SOLR-15016: Replica placement plugins should use container plugins API / configs.
(ab, ilan)
- SOLR-14297: Replace commons-codec Base64 with JDK11 Base64
(Andras Salamon via Houston Putman)
- SOLR-15113: Do not attempt to start Solr server when embedded ZK fails
(Mike Drob)
- SOLR-14067: StatelessScriptUpdateProcessorFactory moved to it's own /module/scripting/ package instead of shipping as
part of Solr due to security concerns. Renamed to ScriptUpdateProcessorFactory for simpler name.
(Eric Pugh)
- SOLR-15118: Switch /v2/collections APIs over to the now-preferred annotated-POJO implementation approach
(Jason Gerlowski)
- SOLR-14928: Allow cluster state updates to be done in a distributed fashion without going through Overseer
(Ilan Ginzburg)
- SOLR-15002: Upgrade httpclient to 4.5.13 and httpcore to 4.4.13
(Andras Salamon via janhoy)
- SOLR-15121: Move XSLT (tr param) response writer and update request handler to scripting module.
(Eric Pugh, David Smiley)
- SOLR-15169: SolrPaths.assertPathAllowed normalization problem
(Andras Salamon via janhoy)
- SOLR-15292: SignatureUpdateProcessorFactory will fail to initialize if it is used in a SolrCloud cluster in a way that is
known to be problematic with multiple replicas.
(hossman)
- SOLR-15258: Consolidate ConfigSet handling (C.R.U.D.) into ConfigSetService to allow for viable
alternative implementations of where ConfigSets come from.
(Nazerke Seidan, David Smiley)
- SOLR-15280: Fixed misleading error message in HdfsDirectoryFactory
(Andras Salamon via janhoy)
- SOLR-15146: Allow Collection API and Config Set API to be done in a distributed fashion without going through Overseer
(Ilan Ginzburg)
- SOLR-15356: "mergeindexes" should not use UninvertingReader (FieldCache).
(David Smiley)
- SOLR-15356: UninvertDocValuesMergePolicy was removed because it's incompatible with Lucene 9.
(David Smiley, Christine Poerschke)
- SOLR-15283: Overhaul distributed-tracing. Solr's "samplePercentage" is gone; tracing is either
enabled and you always have trace IDs or it isn't, based on whether you configure a tracing plugin.
Reporting of traces to a server like Zipkin is a configuration matter of the tracing plugin you
use. The Jaeger plugin is now completely configurable via system properties or environment vars,
not by values in solr.xml.
(David Smiley)
- SOLR-14166: Non-cached filter queries are now pushed down to Lucene, possibly benefiting from
TwoPhaseIterator, which can make a big difference for some queries.
(David Smiley)
- SOLR-15369: PackageStoreAPI will only be loaded in SolrCloud mode
(Mike Drob)
- SOLR-15222: userfiles dir will only be created in SolrCloud mode
(Mike Drob)
- SOLR-15401: Document the new NorwegianNormalizationFilter introduced in LUCENE-9929.
(janhoy)
- SOLR-15409: Upgrade to Zookeeper 3.7.0
(Mike Drob)
- SOLR-15439: Upgrade jose4j library used for JWT authentication processing, to v0.7.7
(janhoy)
- SOLR-15385, SOLR-15535: Address many rawtypes warnings, resulting in several modified signatures in the public API.
(Mike Drob, David Smiley, Christine Poerschke)
- SOLR-15470: The binary distribution no longer contains test-framework jars
(janhoy)
- SOLR-15471: Rename lang id whitelist parameter to allowlist
(Mike Drob)
- SOLR-14341: SolrCloud: move the reference a collection has to a configSet in ZooKeeper from the
collections/collectionName into its state.json. For many-collection clusters, this is an
optimization when the cluster status is fetched.
(Nazerke Seidan, David Smiley)
- SOLR-15517: Remove unnecessary no-op implementation of SolrCoreAware in ExpandComponent and TermVectorComponent.
(Christine Poerschke)
- SOLR-15309: Add missing IntelliJ IDEA entries to the .gitignore file
(Pushkar Raste via Anshum Gupta)
- SOLR-15428: Integrate the OpenJDK JMH micro benchmark framework for micro benchmarks and performance comparisons
and investigation.
(Mark Miller)
- SOLR-15111: Use JDK11 Base64 instead of own implementation
(Andras Salamon via janhoy)
- SOLR-15612: Remove unnecessary https8 jetty module
(Houston Putman)
- SOLR-15606: Rename (deprecated) Insanity.java to NumericHidingLeafReader.java adjusting callers to match.
(Christine Poerschke)
- SOLR-15480: Make Tuple copy constructor, clone and merge consistent w.r.t. markers (EOF, EXCEPTION), field names
and labels.
(John Durham, Mike Drob, Christine Poerschke)
- SOLR-15692: Fix parsing of -zkHost flag for the bin/solr commands
(janhoy)
- SOLR-15699: Removal of the (unused since 5.0) "updateHandler/indexWriter/closeWaitsForMerges" solrconfig.xml element.
(Christine Poerschke)
- SOLR-15728: Remove dead, unused log rotation code from SolrCLI
(janhoy)
- SOLR-15791: Remove remaining unused <admin/> clauses from solrconfigs.
(Eric Pugh)
- SOLR-15784: Remove SolrJ dependency on commons-io.
(Mike Drob)
- SOLR-15809: Get rid of blacklist/whitelist terminology
JWTAuthPlugin parameter 'algWhitelist' is now 'algAllowlist'
Environment variables SOLR_IP_WHITELIST and SOLR_IP_BLACKLIST are no longer supported, but replaced with
SOLR_IP_ALLOWLIST and SOLR_IP_DENYLIST.
(janhoy)
- SOLR-15807: New LogListener class for tests to use to make assertions about what Log messages should or should not
be produced by a test
(hossman)
- SOLR-14660: Move HDFS support to a new HDFS module
(Istvan Farkas, Kevin Risden)
- SOLR-15907: Move JWT Authentication plugin to module 'jwt-auth'
(janhoy)
- SOLR-15845: Add a new SolrVersion class to manage Solr's version independently from the Lucene version we consume
(janhoy)
- SOLR-15912: Upgrade guava to version 31.0.1-jre
(janhoy)
- SOLR-14858: Add the server WEB-INF/lib directory to the classpath for the solr-exporter script. Will allow the script
to work when the dist/solrj-lib jars are missing in the Docker image.
(Houston Putman)
- SOLR-15124: Removed three core level admin API endpoints because they are already registered at the node level where they
really belong: /admin/threads, /admin/properties, /admin/logging
(Nazerke Seidan, David Smiley)
- SOLR-14142: Jetty's RequestLog is enabled by default. If you don't want these logs, you can disable
via SOLR_REQUESTLOG_ENABLED=false.
(rmuir, janhoy)
- SOLR-15924: Remove lucene-libs from module packaging. The lucene libraries will be included in lib/ with all
other dependencies.
(Houston Putman)
- SOLR-15934: CloseHook is now an interface with default method implementations instead of an abstract class.
(Mike Drob)
- SOLR-15953: Remove unused LTRThreadModule.preClose CloseHook logic.
(Christine Poerschke)
- SOLR-15916: Remove dist/ from the binary release. The solr-core and solrj jars will be included in the server
WEB-INF/lib directory, and each individual module's jar will be included in its directory's lib/ folder.
(Houston Putman)
- SOLR-15917: "Contrib Modules" have been renamed to "Modules" and have been moved from "contrib/" to "modules/".
(Houston Putman)
- SOLR-15632: Upgrade to Caffeine Cache 3.0.5
(Mike Drob)
- SOLR-15954: Move the Prometheus Exporter from "solr/modules/prometheus-exporter" to "solr/prometheus-exporter".
(Houston Putman)
- SOLR-15957: Add port and scraping information to Solr Prometheus startup logging.
(Houston Putman)
- SOLR-15936: Reduce unnecessary startup logging, such as SSL warnings when SSL not in use
(janhoy)
- SOLR-15950: Create SOLR_HOME/filestore lazily on first use. Do not automatically create SOLR_HOME/userfiles
(janhoy)
- SOLR-15777: Forbid useDocValuesAsStored for ICUCollationField (warn for luceneMatchVersion < 9.0.0).
(Michael Gibney)
- SOLR-12901: Highlighting: hl.method=unified is the new default.
(David Smiley)
- SOLR-15991: analysis-extras module tests shouldn't rely on log4j dependency
(Kevin Risden)
- SOLR-15064: Atomic/partial updates to nested documents now _require_ the `\_root_` field to
clearly show the document isn't a root document. Solr 8 would fallback on the `\_route_` param but no longer.
(David Smiley)
- SOLR-15943: Simplify HDFS cache to account for LUCENE-10366 / LUCENE-10376
(Kevin Risden)
- SOLR-15942: Upgrade Hadoop to 3.3.1
(Kevin Risden)
- SOLR-15990: Use google-cloud-bom for gcs-repository dependencies
(Kevin Risden)
- SOLR-15989: Upgrade to Tika 1.28.1
(Kevin Risden)
- SOLR-16042: Fix TestSolrCloudWithKerberosAlt.testBasics failure due to minikdc locale
(Kevin Risden)
- SOLR-16043: Fix HDFS tests - "Command processor" thread leak
(Mike Drob, Kevin Risden)
- SOLR-16074: Use PathUtils.deleteDirectory(path) to try to avoid commons-io IO-751
(Houston Putman, Mike Drob, Kevin Risden)
- SOLR-16039: Upgrade to Hadoop 3.3.2
(Kevin Risden)
- SOLR-15223: SolrJ: The project is migrating away from the Apache HttpClient dependency in favor of
Jetty's client (supporting HTTP/2), to occur over the 9x releases. We deprecated HttpSolrClient
and friends in favor of Http2SolrClient and equivalents. For SolrCloud, the former CloudSolrClient
was renamed CloudLegacySolrClient (deprecated), and instead BaseCloudSolrClient was renamed to
CloudSolrClient and given a Builder for the Jetty based HTTP/2 client.
(janhoy, David Smiley)
- SOLR-16061: CloudSolrClient refactoring: Removed ZK specific methods
(Haythem Khiri, David Smiley, janhoy)
- SOLR-16101: Add empty constructor for SolrInputDocument
(Anshum Gupta)
- SOLR-15780: The v2 API is now treated as "experimental". It may change in backwards-incompatible ways in subsequent releases
as it expands to cover additional functionality and prepares to supplant the v1 API.
(Jason Gerlowski)
- Bug Fixes (31)
- SOLR-15849: Fix the connection reset problem caused by the incorrect use of 4LW with \n when monitoring zooKeeper status
(Fa Ming).
- SOLR-14546: Fix for a relatively hard to hit issue in OverseerTaskProcessor that could lead to out of order execution
of Collection API tasks competing for a lock
(Ilan Ginzburg).
- SOLR-15162: Allow readOnly parameter to be used with v2 modify collection command
(Eric Pugh)
- SOLR-15334: Return error response when failing auth in PKIAuthPlugin
(Mike Drob)
- SOLR-15449: Edismax sow incorrectly affect mm parameter in multi field search
(Alessandro Benedetti, Michael Gibney, David Smiley)
- SOLR-15410: Always use -Xverbosegclog for OpenJ9.
(Colvin Cowie via Eric Pugh)
- SOLR-15653: Fix collection creation race that assumes a local clusterstate when the collection has only just been
created.
(Mark Miller)
- SOLR-10529: Solr UI Health Check enable/disable ping Button doesn't work
(Oscar Wang via janhoy)
- SOLR-15783: Prevent Logging MDC values from leaking between request threads, and set 'trace_id' in MDC as soon as it's
available
(hossman)
- SOLR-14781: Removed many unused classes that accumulated over the years.
(Bence Szabo, David Smiley)
- SOLR-15871: Update Log4J to 2.17.1
(weidongkl via janhoy)
- SOLR-15854: Let RealtimeGet component support negative filters
(Tomás Fernández Löbbe)
- SOLR-15501: GCSBackupRepository no longer strictly requires a pointer to a service account JSON file, allowing users
running within GCP to take advantage of it's "Workload Identity" and other role-based access feature.
(Jacek Kikiewicz, Martin Stocker via Jason Gerlowski)
- SOLR-15919: Replace File with Path for many ZK operations
(Mike Drob)
- SOLR-15259: hl.fragAlignRatio now defaults to 0.33 to be faster and maybe looks nicer.
(David Smiley)
- SOLR-15944: The Tagger's JSON response format now always uses an object/map to represent each tag instead of an
array, which didn't make sense.
(David Smiley)
- SOLR-15842: Async response for backups now correctly aggregates and returns information
(Houston Putman, Artem Abeleshev, Christine Poerschke)
- SOLR-14595: Consistent overrequest across different facet methods for `sort:index` JSON Facet field
(Michael Gibney, hossman)
- SOLR-15961: Fix bug in PKIAuthenticationPlugin that can cause a request to fail with 401 Unauthorized instead
of re-fetching expired remote keys from other nodes.
(Tomás Fernández Löbbe)
- SOLR-15968: Hide annoying WARN log from bin/solr zk command
(janhoy, Mike Drob)
- SOLR-16075: ShowFileHandler path parameter is now validated to be relative to instance conf dir in standalone mode
(janhoy)
- SOLR-15558: Don't wait for zombie processes to exit when stopping.
(Colvin Cowie)
- SOLR-16019: UTF-8 parsing errors for parameters should cause a HTTP 400 status code, not 500
(janhoy, Matthias Pigulla)
- SOLR-15983: Fix ClassCastException in UpdateLog$LogReplayer.doReplay.
(Christine Poerschke, David Smiley)
- SOLR-16090: Better error message when JWT auth SIGNATURE_INVALID during token parsing
(janhoy)
- SOLR-15333: Reduced spurious warn logging by AbstractSpatialPrefixTreeFieldType field properties
(Steffen Moldenhauer, David Smiley, Mike Drob)
- SOLR-15405: Make HDFS handle little endian LUCENE-9047
(Kevin Risden)
- SOLR-16009: Force Calcite's Rel simplify config flag to false to avoid erasing filters that are meaningful to Solr,
but look like nonsense to Calcite, such as AND'd filters on the same multi-valued field
(Timothy Potter, Kiran Chitturi)
- SOLR-16022: Enforce special character requirements on passwords with length less than 15
(Timothy Potter)
- SOLR-16143: SolrConfig can miss updates from ZooKeeper when deleting and recreating file items
(Mike Drob)
- SOLR-16164: ConfigSet API returns error if untrusted user creates from _default configset
(Eric Pugh, Kevin Risden)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- New Features (1)
- SOLR-15694, 15715: Node roles and dedicated query coordinator nodes
(Hitesh Khamesra, Ishan Chattopadhyaya, noble)
- Bug Fixes (14)
- SOLR-16429: Add AWS STS support in s3-repository
(Josh Souza via Houston Putman and Kevin Risden)
- SOLR-14679: Fix continuously growing TLOGs on TLOG replicas
(Viktor Molnár via Houston Putman)
- SOLR-16293: Luke request fails for document with a binary field
(Kevin Risden)
- SOLR-16477: Collection RENAME api creates broken alias
(Alex Deparvu via Kevin Risden)
- SOLR-16274: HEAD request for managed resource returns 500 Server Error
(Kevin Risden)
- SOLR-13219: Fix NPE in FieldLengthFeature with non-stored/missing fields
(Nick Veenhof, Tomasz Elendt)
- SOLR-16984: Add missing sysProps to redaction logic
(Houston Putman)
- SOLR-16165: Rare Deadlock in SlotAcc initialization
(Justin Sweeney, noble)
- SOLR-16110: Using Schema/Config API breaks the File-Upload of Config Set File
(Steffen Moldenhauer, Kevin Risden)
- SOLR-16451: Don't fetch the PRS states while registering the collection watch
(Hitesh Khamesra via Ishan Chattopadhyaya)
- SOLR-16452: Do not update PRS states if local version is newer
(Hitesh Khamesra via noble)
- SOLR-16777: Schema Designer now correctly manages trust of the ConfigSets it is managing.
(Ishan Chattopadhyaya, Skay, Houston Putman)
- SOLR-17098: ZK Credentials and ACLs are no longer sent to all ZK Servers when using Streaming Expressions.
They will only be used when sent to the default ZK Host.
(Houston Putman, Jan Høydahl, David Smiley, Gus Heck, Qing Xu)
- SOLR-17120: Fix NullPointerException in UpdateLog.applyOlderUpdates that can occur if there are multiple partial
updates of the same document in separate requests using commitWithin.
(Calvin Smith, Christine Poerschke)
- Optimizations (2)
- SOLR-16555: SolrIndexSearcher - FilterCache intersections/andNot should not clone bitsets repeatedly
(Kevin Risden, David Smiley)
- SOLR-17004: ZkStateReader waitForState should check clusterState before using watchers
(Kevin Risden)
- Other Changes (9)
- SOLR-16141: Upgrade Apache Tika to 1.28.4
(Kevin Risden)
- SOLR-16324: Upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9
(Kevin Risden)
- SOLR-16332: Upgrade Jetty to 9.4.48.v20220622
(Kevin Risden)
- SOLR-16626, SOLR-16424: Upgrade to Netty 4.1.87.Final
(Kevin Risden, janhoy)
- SOLR-16421: Upgrade Apache Calcite to 1.32.0 and Avatica to 1.22.0
(Kevin Risden)
- SOLR-16464: Upgrade commons-text to 1.10.0
(Kevin Risden)
- SOLR-17025: Upgrade Jetty to 9.4.53.v20231009
(Kevin Risden)
- SOLR-14853: Security: Converted enableRemoteStreaming and enableStreamBody solrconfig options into system properties and env vars.
Attempts to set them the old way are no-op and log a warning.
(David Smiley, janhoy, Ishan Chattopadhyaya)
- SOLR-16949: Restrict certain file types from being uploaded to or downloaded from Config Sets
(janhoy, Houston Putman)
- Bug Fixes (27)
- SOLR-15871: Update Log4J to 2.17.1
(weidongkl via janhoy)
- SOLR-15854: Let RealtimeGet component support negative filters
(Tomás Fernández Löbbe)
- SOLR-15961: Fix bug in PKIAuthenticationPlugin that can cause a request to fail with 401 Unauthorized instead
of re-fetching expired remote keys from other nodes.
(Tomás Fernández Löbbe)
- SOLR-15558: Don't wait for zombie processes to exit when stopping.
(Colvin Cowie)
- SOLR-15587: Don't use the UrlScheme singleton on the client-side to determine cluster's scheme,
only get from ClusterState provider impl
(Timothy Potter)
- SOLR-14569: Configuring a shardHandlerFactory on the /select requestHandler results in HTTP 401 when searching on alias
in secured Solr.
(Isabelle Giguere, Jason Gerlowski, Anshum Gupta, Mark Mark Miller)
- SOLR-15974: Remove Calcite's ENUMERABLE_AGGREGATE_RULE as Solr only supports push-down for LogicalAggregate
(Timothy Potter, Kiran Chitturi)
- SOLR-16009: Force Calcite's Rel simplify config flag to false to avoid erasing filters that are meaningful to Solr,
but look like nonsense to Calcite, such as AND'd filters on the same multi-valued field
(Timothy Potter, Kiran Chitturi)
- SOLR-16095: Upgrade JQuery to 3.5.1 in velocity contrib module
(Bence Szabo via Jan Høydahl)
- SOLR-16022: Enforce special character requirements on passwords with length less than 15
(Timothy Potter)
- SOLR-15849: Fix the connection reset problem caused by the incorrect use of 4LW with \n when monitoring zooKeeper status
(Fa Ming).
- SOLR-15116: Support a HEAD request for managed resources.
(Eric Pugh)
- SOLR-16089: Fix empty columns in Cloud Node UI Screen when replica is in down state.
(hossman)
- SOLR-16145: Fix very rare NPE in SolrCloud rejoinOverseerElection.
(David Smiley)
- SOLR-16168: Spellcheck NPE if invalid dictionary name is provided
(Kevin Risden)
- SOLR-15968: Hide annoying WARN log from bin/solr zk command
(janhoy, Mike Drob)
- SOLR-16075: ShowFileHandler path parameter is now validated to be relative to instance conf dir in standalone mode
(janhoy)
- SOLR-16019: UTF-8 parsing errors for parameters should cause a HTTP 400 status code, not 500
(janhoy, Matthias Pigulla)
- SOLR-16143: SolrConfig can miss updates from ZooKeeper when deleting and recreating file items
(Mike Drob)
- SOLR-16199: Improve query syntax construction for SQL LIKE clause with phrases and wildcards
(Kiran Chitturi, Aroop Ganguly, Amrit Sarkar via Timothy Potter)
- SOLR-16191: Validate that installed ps utility supports -p flag, so that we do not inadvertantly stop the wrong process.
(Mike Drob, Michael Gibney)
- SOLR-16213: Upgrade Jackson to version 2.13.3
(janhoy)
- SOLR-16215: Escape query characters in Solr SQL Array UDF functions
(Kiran Chitturi via Timothy Potter)
- SOLR-15965: Use better signatures for the PKI Authentication plugin.
(Mike Drob)
- SOLR-16209: Rolling restart will no longer trigger as much PKI Plugin error logging.
(Mike Drob, Tomás Fernández Löbbe)
- SOLR-16227: Solr SQL query parsing fails on multiple LIKE clauses and multiple terms
(Kiran Chitturi via David Smiley)
- SOLR-16218: failOnVersionConflicts option not working for in-place updates
(Lamine Idjeraoui, Houston Putman, Anshum Gupta)
- Bug Fixes (19)
- SOLR-15635: Don't close hooks twice when SolrRequestInfo is cleared twice; or /export with classic join
closed fromCore if provided
(Mikhail Khludnev, David Smiley)
- SOLR-15795: Fix REPLACENODE to not use source node when choosing a target node for new replicas
(Houston Putman)
- SOLR-15804: Admin UI once again can show files in the Core/Collection -> Files screen. Fixed regression from 8.11.0
(Karl Stoney, janhoy)
- SOLR-15768: Set default permissions for the zookeeper endpoints so users won't need to configure custom permissions for these
(janhoy)
- SOLR-15774: Avoid weird off-by-one errors with Angular's 'chosen' select box directive for the security and schema-designer screens in Admin UI
(Timothy Potter)
- SOLR-13900: Reset index values on authorization rules after deleting by index
(Timothy Potter)
- SOLR-15794: Switching a PRS collection from true -> false -> true results in INACTIVE replicas
(noble)
- SOLR-15813: Schema designer not handling `update.autoCreateFields` stored as a string (vs. boolean) in the config overlay
(Timothy Potter)
- SOLR-15825: Security UI 'hasPermission' check should check if the user has the "all" permission if the requested permission is not defined
to match how the backend works
(Timothy Potter)
- SOLR-15826: ResourceLoader should better respect allowed paths
(janhoy)
- SOLR-15828: AuthTool (in SolrCLI) should include the config-read, collection-admin-read, core-admin-read, and all permissions in the initial security.json
to avoid warnings in the security UI
(Timothy Potter)
- SOLR-15199: Pass thru all command actions not explicitly handled by the bin/solr script, such as 'api', to the SolrCLI Java app
(Timothy Potter)
- SOLR-15833: Enable exists queries on spatial types.
(Mike Drob, Houston Putman)
- SOLR-15832: Clean-up after publish action in Schema Designer shouldn't fail if .system collection doesn't exist
(Timothy Potter)
- SOLR-15844: Upgrade Velocity from v2.0 to v2.3
(wcmrnd1, janhoy)
- SOLR-8319: Fix NPE in pivot facets, add non-Analyzed query method in FieldType.
(Houston Putman, Isabelle Giguere)
- SOLR-15843: Update Log4J to 2.16
(Mike Drob, janhoy)
- SOLR-15324: Upgrade jaegertracing to 1.6.0 and libthrift to 0.14.1
(wcmrnd1, janhoy)
- SOLR-15405: Make HDFS handle little endian LUCENE-9047
(Kevin Risden)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- New Features (4)
- SOLR-7642: opt-in support to create ZK chroot on startup
(Timothy Potter, Shawn Heisey, Mark Miller, Tomas Eduardo Fernandez Lobbe,
Jan Høydahl, Steve Molloy, Isabelle Giguere, David Eric Pugh, Gus Heck, Christine Poerschke, Houston Putman)
- SOLR-15708: SolrJ support for ConfigSet uploading
(ab, hossman)
- SOLR-12666: Add authn & authz plugins that supports multiple authentication schemes, such as Bearer and Basic
(Timothy Potter, janhoy)
- SOLR-15721: Support editing Basic auth config from the security UI when using the MultiAuthPlugin
(Timothy Potter)
- Improvements (2)
- SOLR-15707: Add the ability to set the AWS Profile per S3Repository
(Houston Putman)
- SOLR-15700: Make include paths in bin/solr consistent
(turnertg, janhoy via Eric Pugh)
- Optimizations (1)
- (No changes)
- Bug Fixes (9)
- SOLR-15626: The "config-read" permission has been fixed to properly allow access to `/solr/admin/configs?action=LIST`
(Jon Senchyna via Eric Pugh)
- SOLR-15691: Admin UI raises yellow warning even when only case of hostnames differ
(janhoy)
- SOLR-15702: Fix S3Repository to follow BackupRepository.createDirectory() API contract
(Houston Putman)
- SOLR-15696: Incremental backups no longer fail on collections that had previously seen a 'SPLITSHARD' operation.
(Jason Gerlowski)
- SOLR-15628: The SolrException.log() helper method has been fixed to correctly passes the Throwable to the Logger w/o stringification
(hossman)
- SOLR-15676: Fix PeerSync failure due to RealTimeGetComponent returning duplicates.
(Ramsey Haddad, Christine Poerschke, David Smiley)
- SOLR-15722: Delete Replica does not delete the Per replica state
(noble)
- SOLR-15766: MultiAuthPlugin should send non-AJAX anonymous requests to the plugin that allows anonymous requests
(Timothy Potter, Eric Pugh)
- SOLR-15706: Incremental backup 'purges' no longer delete all index files(!)
(Jason Gerlowski)
- Build (1)
- LUCENE-10104, SOLR-15631: Upgrade forbiddenapis to version 3.2.
(Uwe Schindler)
- Other Changes (3)
- SOLR-15678: Allow only known content types in ShowFileRequestHandler
(janhoy, Gus Heck, Mal Aware)
- SOLR-15697: Add ErrorLogMuter to test-framework, deprecating SolrException.ignorePatterns
(hossman)
- SOLR-15761: Remove JdbcStream from Lang
(Joel Bernstein)
- Other Changes (1)
- SOLR-15677: Upgrade the following dependencies with vulnerabilities.
-
jetty: 9.4.44.v20210927
-
tika: 1.27
-
commons-compress: 1.21
-
netty: 4.1.68.Final
-
fasterxml.jackson: 2.12.3
-
errorprone: 2.9.0
-
gcp-client: 1.32.1
(Houston Putman)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- New Features (5)
- SOLR-15472: New shards.preference option for preferring replicas based on their leader status, i.e.
`shards.preference=replica.leader:false` would prefer non-leader replicas.
(wei wang via Timothy Potter)
- SOLR-15208: Add the countDist aggregation to the stats, facet and timeseries Streaming Expressions
(Joel Bernstein)
- SOLR-15277: Schema Designer UI
(Timothy Potter, Kiran Chitturi)
- SOLR-15527: Security screen in Admin UI for managing users, roles, and permissions
(Timothy Potter)
- SOLR-15089: Allow backup/restoration to Amazon's S3 blobstore
(Andy Throgmorton, Houston Putman, Jason Gerlowski, Pierre Salagnac, Mike Drob)
- Improvements (20)
- SOLR-15460: Implement LIKE, IS NOT NULL, IS NULL, and support wildcard * in equals string literal for Parallel SQL
(Timothy Potter, Houston Putman)
- SOLR-15451: SolrSchema (for Parallel SQL) should use PKI principal for internal request to /admin/luke to get table metadata
(Timothy Potter)
- SOLR-15456: Get field type info from luke for custom fields instead of defaulting to String in Parallel SQL
(Timothy Potter)
- SOLR-15461: Upgrade Apache Calcite to 1.27.0.
(Mark Miller, Timothy Potter)
- SOLR-15489: Implement OFFSET & FETCH for LIMIT SQL queries
(Timothy Potter)
- SOLR-15475: Implement COUNT and APPROX_COUNT_DISTINCT aggregation functions for Parallel SQL
(Timothy Potter)
- SOLR-15499: StatsStream implement ParallelMetricsRollup to allow for tiered computation of SQL metrics
over collection aliases backed by many collections, potentially with many shards in each
(Timothy Potter)
- SOLR-15525: Read ZK credentials from a file specified using a system property instead of exposing plain-text
credentials as system properties
(Timothy Potter)
- SOLR-15549: ZkStateReader now supports connecting to 9.0 Solr Clouds
(Houston Putman)
- SOLR-15570: Include fields declared in the schema in table metadata (SQL) even if they are empty
(Timothy Potter)
- SOLR-15564: Improve filtering expressions in /admin/metrics.
(ab)
- SOLR-15576: Allow filtering on ISO-8601 formatted timestamp literals in SQL WHERE clause
(Timothy Potter)
- SOLR-9853: Project multi-valued fields in SQL query results
(Timothy Potter)
- SOLR-15599: Upgrade AWS SDK from v1 to v2 for S3 Repository
(Houston Putman)
- SOLR-15609: Upgrade log4j to 2.14.1
(hossman)
- SOLR-11752: Support "Accept-Encoding" header to enable response gzip compression
(Matthew Sporleder, Houston Putman, Shawn Heisey, Jan Høydahl)
- SOLR-15610: log4j-layout-template-json is now included in solr server, allowing JSON formatted logs to be configurable.
(hossman)
- SOLR-15607: Explicitly indent results in Admin Query UI by default
(Olivia Falk via Eric Pugh)
- SOLR-15555: Identical filter queries will no longer race to record a filterCache entry. Allows for async cache computation instead of
all queries contending for the same set of locks on the cache. Fixed a bug with ramBytes usage reporting on filterCache.
Upgraded Caffeine Cache to 2.9.2
(Mike Drob)
- SOLR-15572: Improve the default Prometheus exporter config performance.
(ab)
- Optimizations (2)
- SOLR-15433: Replace transient core cache LRU by Caffeine cache.
(Bruno Roustant)
- SOLR-15496: Reuse comparator objects in contrib/ltr rescorer classes.
(marcosfpr via Christine Poerschke)
- Bug Fixes (13)
- SOLR-15311: Support async parameter in MODIFYCOLLECTION
(Nazerke Seidan, Christine Poerschke, David Smiley)
- SOLR-15531: SignificantTerms streaming function should not fail on empty collections
(Benedikt Arnold via Mike Drob)
- SOLR-15482: Cross-collection join fixed to ignore documents that do not contain value in "from" field
(hossman)
- SOLR-14506: Fix NPE in COLSTATUS for collections using implicit routing.
(Austin Weidler, ab)
- SOLR-15575: Propagate request level basic auth creds from the top-level async CollectionAdminRequest to internally
used async requests, such as async status checking
(Timothy Potter)
- SOLR-14758: Fix NPE in QueryComponent.mergeIds when using timeAllowed and sorting
(Bram Van Dam, Uwe Schindler)
- SOLR-15579: Allow for many values in a SQL IN clause; previously using more than 19 would result in
the IN clause being ignored, now users are only limited by the `maxBooleanClauses` configured for a collection
(Timothy Potter)
- SOLR-8889: Fixed various problems in Solr and SolrJ that could cause deleteById commands with "_route_" information to processed
by the wrong shard, and/or fail when forwarded to replicas from the shard leader.
(Dan Fox, Yuki Yano, hossman)
- SOLR-13209: Fix NPE when no grouping query specified
(Collins Abanda via Mike Drob)
- SOLR-6156: Fix NullPointerException if group.field grouping is used with rows=0 and timeAllowed.
(Modassar Ather, Christine Poerschke)
- SOLR-14457: SolrClient leaks connections on compressed responses if the response is malformed
(Houston Putman)
- SOLR-15595: Partial results from shard queries needlessly discarded for queries without sort field
(Michael Kosten via Mike Drob, Eric Pugh)
- SOLR-15620: Download Config button in Schema Designer screen should not require user to re-login when already authenticated
(Timothy Potter)
- Other Changes (5)
- SOLR-15566: Clarify ref guide documentation about SQL queries with `SELECT *` requiring a `LIMIT` clause
(Timothy Potter)
- SOLR-15573: bin/solr auth utility should provide role bindings for `security-read` and `config-edit` by default
to protect the security and schema designer screens in the Admin UI
(Timothy Potter)
- SOLR-15486: During node shutdown pausing of updates and waiting for in-flight update requests to finish
before closing cores is no longer SolrCloud specific.
(Christine Poerschke, David Smiley)
- SOLR-15599: woodstox-core-asl:4.4.1 (org.codehaus) replaced with woodstox-core:6.2.4 (com.fasterxml)
(Houston Putman)
- SOLR-15621: index.html for Admin UI should send Solr version in the request for JavaScript files
(Timothy Potter)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- New Features (8)
- SOLR-14330: ExpandComponent now supports an expand.nullGroup=true option
(hossman)
- SOLR-15150: New update.partial.requireInPlace=true option to prevent any partial document updates that can't be done In-Place
(hossman)
- SOLR-15087: SolrCloud backups can now be restored to compatible existing collections, instead of requiring a new collection
for each restore.
(Jason Gerlowski, shalin, Cao Manh Dat)
- SOLR-15132: Add temporal graph query to the nodes Streaming Expression
(Joel Bernstein)
- SOLR-15365: Improved Grafana dashboard for Prometheus Exporter with new Solr Cluster row
(janhoy)
- SOLR-15397: Expose zookeeper status in the Prometheus exporter
(janhoy)
- SOLR-15090: A new 'gcs-repository' contrib can be used to store and retrieve backups from Google Cloud Storage.
(Jason Gerlowski, Shalin Mangar, Cao Manh Dat)
- SOLR-12697: In contrib/ltr FieldValueFeature support "stored=false docValues=true" a.k.a. pure DocValues fields.
(Stanislav Livotov, Erick Erickson, Tobias Kässmann, Tom Gilke, Christine Poerschke)
- Improvements (15)
- SOLR-15081: Metrics for a core: add SolrCloud "isLeader" and "replicaState".
(David Smiley)
- SOLR-15085: Prevent EmbeddedSolrServer calling shutdown on a CoreContainer that was passed to it.
(Tim Owen via Mike Drob)
- SOLR-14234: Unhelpful message in RemoteExecutionException.
(ab)
- SOLR-13608: Backups are now done incrementally by default. Multiple backups can be stored at the same location, and each
backup will only upload those files that are new since the last backup.
(Jason Gerlowski, shalin, Cao Manh Dat)
- SOLR-15123: Revamp SolrCLI tool's help descriptions for all commands for consistency and clarity.
(Eric Pugh)
- SOLR-15101: Add "list" and "delete" APIs for managing incremental backups
(Jason Gerlowski, shalin, Cao Manh Dat)
- SOLR-15038: Add elevateOnlyDocsMatchingQuery and collectElevatedDocsWhenCollapsing parameters to query elevation.
(Dennis Berger, Tobias Kässmann via Bruno Roustant)
- SOLR-15191: Enhance hash method of JSON faceting to support EnumFieldType and perhaps some other/custom field types
too.
(Thomas Wöckinger, David Smiley)
- SOLR-15194: Allow Solr to make outbound non SSL calls to a JWT IDP via -Dsolr.auth.jwt.allowOutboundHttp=true property.
(Eric Pugh)
- SOLR-15154: Let Http2SolrClient pass Basic Auth credentials to all requests
(Tomás Fernández Löbbe)
- SOLR-6152: Pre-populating values into search parameters on the query page of solr admin
(jafurrer, daniel6691)
- SOLR-11233: Add optional JAVA8_GC_LOG_FILE_OPTS for bin/solr.
(Pranav Murugappan, Christine Poerschke)
- SOLR-15155: Let CloudHttp2SolrClient accept an external Http2SolrClient Builder
(Tomás Fernández Löbbe)
- SOLR-15156: [child] doc transformer's childFilter param no longer applies query syntax escaping.
It was inconsistent with other Solr options. It did this escaping since 8.0 but not prior.
(David Smiley)
- SOLR-15337: Avoid XPath in solrconfig.xml parsing
(noble)
- Optimizations (4)
- SOLR-15079: Block Collapse - Faster collapse code when groups are co-located via Block Join style nested doc indexing.
Used by default when field=_root_, or explicitly requested for other fields via hint=block.
(Joel Bernstein, hossman)
- SOLR-15120 Reduce duplicated core creation work
(Mike Drob, Mark Miller)
- SOLR-15067 ChildDocTransformerFactory uses perSegFilter cache
(Bence Szabo via David Smiley)
- SOLR-15315: Optimise contrib/ltr MultipleAdditiveTreesModel by changing recursion to loop in tree traversal.
(Suryakant Pandey, Alessandro Benedetti, Christine Poerschke)
- Bug Fixes (14)
- SOLR-15078: Fix ExpandComponent behavior when expanding on numeric fields to differentiate '0' group from null group
(hossman)
- SOLR-15149: Better exception handling for LTR model creation errors
(Alessandro Benedetti, Christine Poerschke)
- SOLR-13034: Partial (AKA Atomic) updates could encounter "LazyField" instances in the document
cache and not know hot to deal with them when writing the updated doc to the update log.
(David Smiley)
- SOLR-15191: Fix JSON Faceting on EnumFieldType if allBuckets, numBuckets or missing is set.
(Thomas Wöckinger via David Smiley)
- SOLR-15457: Fix JSON Faceting on EnumFieldType -- returned values should not be internal ordinals.
(Thomas Wöckinger via David Smiley)
- SOLR-15216: Fix for Invalid Reference to data.followers in Admin UI.
(Dean Pearce)
- SOLR-15273: Fix NullPointerException in StoredFieldsShardResponseProcessor that happened when a field name alias is
used for the unique key field in searches with distributed result grouping.
(limingnihao via Christine Poerschke)
- SOLR-15384: Zookeeper Status handler /admin/zookeeper/status not queryable from SolrJ
(janhoy)
- SOLR-11921: Move "cursorMark" logic from QueryComponent to SearchHandler so it can work with things like QueryElevationComponent
that modify the SortSpec in prepare(), as well as possible custom "search" components other then QueryComponent.
(hossman)
- SOLR-15383: Solr Zookeeper status page shows green even when some Zookeepers are not serving requests
(janhoy)
- SOLR-11904: Mark ReplicationHandler's polling thread as a Solr server thread so the PKI Interceptor is activated to
allow PULL replicas to replicate from security-enabled leaders
(Timothy Potter, Torsten Bøgh Köster)
- SOLR-15418: The V2 API threw errors for GET requests to collection handlers like /select.
(David Smiley)
- SOLR-15424: Solr replication UI wraps ETA time on top of next line
(janhoy)
- SOLR-15399: IndexFetcher should not issue a local commit for PULL replicas when leader's version is zero
(Timothy Potter)
- Other Changes (6)
- SOLR-15118: Deprecate CollectionAdminRequest.getV2Request().
(Jason Gerlowski)
- SOLR-15292: An ERROR is logged if SignatureUpdateProcessorFactory is used in SolrCloud cluster in a way that is known to
be problematic with multiple replicas. In 9.0 this situation will prevent collection initialization.
(hossman)
- SOLR-15377: Improve IndexFetcher's exception logging upon replication errors.
(Torsten Bøgh Köster, Christine Poerschke)
- SOLR-15416: Metrics history handler and metrics history collection (together with its
specialized reporters) have been deprecated and will be removed in 9.0.
(ab)
- SOLR-14978: Enable OOM Killer Script in Solr Foreground. Simplify getting heap dumps on OOM.
(Mike Drob, Houston Putman)
- SOLR-15316: Upgrade Jetty to 9.4.41.v20210516
(janhoy, Mike Drob)
- Bug Fixes (4)
- SOLR-15249: Properly set ZK ACLs on /security.json
(Mike Drob)
- SOLR-15288: Hardening NODEDOWN event in PRS collections
(noble)
- SOLR-15233: Set doAs param in ConfigurableInternodeAuthHadoopPlugin
(Geza Nagy, Jason Gerlowski, Mike Drob)
- SOLR-15217: Use shardsWhitelist in ReplicationHandler.
(Bruno Roustant)
- Bug Fixes (5)
- SOLR-15145: System property to control whether base_url is stored in state.json to enable back-compat with older SolrJ versions.
(Timothy Potter)
- SOLR-15114: Fix bug that caused WAND optimization to be disabled in cases where the max score is requested (such as
multi-shard requests in SolrCloud)
(Naoto Minami via Tomás Fernández Löbbe)
- SOLR-15136: Reduce excessive logging introduced with Per Replica States feature
(Ishan Chattopadhyaya)
- SOLR-15138: Collection creation for PerReplicaStates does not scale to large collections as well as regular collections
(Mike Drob, Ilan Ginzburg, noble, Ishan Chattopadhyaya)
- SOLR-15135: Admin UI display collections with perReplicaState=true in graph view
(Timothy Potter)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Upgrade Notes (1)
- Internal logic for identifying 'Solr Home' has been refactored to make testing less error prone. Plugin
developers using SolrPaths.locateSolrHome() or 'new SolrResourceLoader' should check deprecation warnings as existing
some existing functionality will be removed in 9.0. See SOLR-14934 for more details.
- New Features (3)
- SOLR-14907: Add v2 API for configSet upload, including single-file insertion.
(Houston Putman)
- SOLR-14560: Add interleaving support in Learning To Rank.
(Alessandro Benedetti, Christine Poerschke)
- SOLR-15052: Reducing overseer bottlenecks using per-replica states
(noble, Ishan Chattopadhyaya)
- Improvements (16)
- SOLR-14942: Reduce leader election time on node shutdown by removing election nodes before closing cores.
(Cao Manh Dat, Mike Drob, hossman, shalin)
- SOLR-14955: Add env var options for the Prometheus Exporter bin scripts
(Houston Putman)
- SOLR-14683: Metrics API should ensure consistent placeholders for missing values.
(ab)
- SOLR-14977 : ContainerPlugins should be configurable with custom objects
(noble, ab)
- SOLR-15015 : Add interleaving algorithm parameter support in Learning To Rank
(Alessandro Benedetti)
- SOLR-8673: Make JSON Facets extendable
(Tim Owen via Mikhail Khludnev)
- SOLR-14965: metrics: Adds two metrics to the SolrCloud Overseer: solr_metrics_overseer_stateUpdateQueueSize
and solr_metrics_overseer_collectionWorkQueueSize with corresponding entries in the the Prometheus exporter's
default/stock configuration.
(Saatchi Bhalla, Megan Carey, Andrzej Białecki, David Smiley)
- SOLR-14827: Refactor schema loading to not use XPath
(noble)
- SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using CloudSolrStream
(Timothy Potter)
- SOLR-14413: Allow timeAllowed and cursorMark parameters together.
(John Gallagher via Mike Drob)
- SOLR-15062: /api/cluster/zk/ls should give the stat of the current node
(noble)
- SOLR-15069: [child]: the parentFilter parameter is now fully optional and perhaps obsolete.
(David Smiley)
- SOLR-14923: The use case of atomic/partial updates to child documents no longer requires that
_root_ be stored or have docValues, and no longer requires the _nest_path_ field. However it
now requires that the client pass a _root_ field on these updates to point to the root ID.
(David Smiley)
- SOLR-15059: Add panels to the default Grafana dashboard for query performance monitoring, includes updates
to the Prometheus exporter to export query performance metrics, such as QPS and p95.
(Timothy Potter)
- SOLR-15040: Improvements to postlogs timestamp handling
(Joel Bernstein)
- SOLR-14155: Load all other SolrCore plugins from packages
(noble)
- Optimizations (4)
- SOLR-14975: Optimize CoreContainer.getAllCoreNames, getLoadedCoreNames and getCoreDescriptors.
(Bruno Roustant)
- SOLR-15049: Optimize same-core, same-field joins in TopLevelJoinQuery
(Jason Gerlowski)
- SOLR-14923: Indexing nested documents is faster, especially under concurrent indexing load. In addition,
Partial updates to nested documents and Realtime Get of child documents is now more reliable.
(David Smiley, Thomas Wöckinger)
- SOLR-15036: Automatically wrap a facet expression with a select / rollup / sort / plist when using a
collection alias with multiple collections and count|sum|min|max|avg metrics.
(Timothy Potter)
- Bug Fixes (28)
- SOLR-14946: Fix responseHeader being returned in response when omitHeader=true and EmbeddedSolrServer is used
as the client
(Munendra S N)
- SOLR-14940: ReplicationHandler memory leak through SolrCore.closeHooks with unstable ZK connection.
(Anver Sotnikov, Mike Drob)
- SOLR-14969: Prevent creating multiple cores with the same name which leads to instabilities (race condition)
(Erick Erickson, Andreas Hubold)
- SOLR-14961: Fix for deleting zookeeper nodes with same path length. Only the first zk-node was removed.
(Michael Aleythe via Mike Drob)
- SOLR-14971: AtomicUpdate 'remove', 'add-distinct' operations now works on numeric, date fields in uncommitted docs
(Jason Gerlowski)
- SOLR-14983: Fix response returning original score instead of reranked score due to query and filter combining.
(Krishan Goyal, Jason Baik, Christine Poerschke)
- SOLR-14993: Unable to download zookeeper files of 1byte in size
(Erick Erickson, Allen Sooredoo)
- SOLR-14851: Http2SolrClient doesn't handle keystore type correctly
(Andras Salamon via janhoy)
- SOLR-15009: Correctly propogate exceptions from DirectoryFactory.exists
(Mike Drob)
- SOLR-15017: Core lib directories were not being recognized unless the solrconfig included a <lib> directive.
(Thomas Mortagne)
- SOLR-14934: Refactored duplicate "Solr Home" logic into a single place to eliminate risk of tests using divergent values
for a single solr node.
(hossman)
- SOLR-12182: Don't persist base_url in ZK as the URL scheme is variable, compute from node_name instead when reading
state back from ZK.
(Timothy Potter)
- SOLR-15046: Check if SOLR_SSL_ENABLED strictly equal to true for setting solr.jetty.https.port
(Timothy Potter)
- SOLR-14939: JSON range faceting to support cache=false parameter.
(Christine Poerschke, Mike Drob)
- SOLR-15031: Fix preventing null being wrapped in a QueryValueSource subQuery. Such null queries can be caused by query text
resulting in an empty token stream.
(Pieter van Boxtel via Mike Drob)
- SOLR-15061: Fix NPE in SearchHandler when shards.info and no QueryComponent configured.
(Bruno Roustant)
- SOLR-14571: Download speed was hard-coded in the "Replication" screen of the Solr Admin UI.
(Florin Babes via Christine Poerschke)
- SOLR-15048: Fixed collapse parser behavior when dealing with docs boosted by QueryElevationComponent that are in the
null group to treat them consistently regardless of collapse field type or group head selector.
(hossman)
- SOLR-15058: Enforce node_name contains colon and port and find first underscore after colon to parse context
when converting a node_name to a base URL.
(Timothy Potter, Su Sasa)
- SOLR-15047: Fix collapse parser behavior when collapsing on numeric fields to differentiate '0' group from null group
(hossman)
- SOLR-10860: Return proper error code for bad input in case of inplace updates
(Tomas Eduardo Fernandez Lobbe, Munendra S N)
- SOLR-14950: Fix error in copyField regeneration when explicit source/destination is not present in schema but
matches the dynamic rule. The copyFields are rebuilt with replace-field and replace-field-type, if required
(Andrew Shumway, Munendra S N)
- SOLR-12539: Handle parsing of values for 'other', 'include', and 'excludeTags' in JSON facets when specified as
comma-separated values with extra spaces
(hossman, Munendra S N)
- SOLR-14514: Avoid picking 'stream' method in JSON facet when any of 'allBuckets', 'numBuckets', and 'missing' parameters are enabled
(hossman, Munendra S N)
- SOLR-12559: Fix placeholder valuesource 'FIELDNAME' not working with de-referenced values in JSON aggregator parsing.
(hossman, Munendra S N)
- SOLR-15070: Suggester requests made with SolrJ can now use XMLResponseParser
(Jason Gerlowski)
- SOLR-15073: Fix ClassCastException in SystemInfoHandler.getSecurityInfo.
(Nikolay Ivanov, Christine Poerschke)
- SOLR-15071: Fix ArrayIndexOutOfBoundsException in contrib/ltr SolrFeatureScorer.
(Florin Babes, Ovidiu Mihalcea, David Smiley, Christine Poerschke)
- Other Changes (8)
- SOLR-14954: Heavily edit reindexing.adoc
(Sameul García Martínez and Erick Erickson)
- SOLR-14981: Removed search results clustering contrib from 8x (due to Carrot2 incompatibility with Java 1.8).
(Dawid Weiss)
- SOLR-14995: Update Jetty to 9.4.34
(Mike Drob)
- SOLR-14958: Refactor zkHost config logic to make testing easier and reduce risk of incorrect value being used
(hossman)
- SOLR-14951: Upgrade Angular JS 1.7.9 to 1.8.0
(Kevin Risden)
- SOLR-15057: Avoid unnecessary object retention in FacetRangeProcessor.
(Christine Poerschke)
- SOLR-14999: New option for specifying the advertised Solr Port (hostPort). Still defaults to "jetty.port".
(Houston Putman)
- SOLR-15010: Thread dump using jattach if available and jstack is missing.
(Eric Pugh)
- Consult the lucene/CHANGES.txt file for additional, low level, changes in this release.
- New Features (13)
- SOLR-14151 Make schema components load from packages
(noble)
- SOLR-14681: Introduce ability to delete .jar stored in the Package Store.
(MarcusSorealheis , Mike Drob)
- SOLR-14604: Add the ability to uninstall a package from with the Package CLI.
(MarcusSorealheis)
- SOLR-14582: Expose IWC.setMaxCommitMergeWaitMillis in Solr's index config. This is an expert config option that can be
set when using a custom MergePolicy (doesn't have any effect on the default MP)
(Tomás Fernández Löbbe)
- SOLR-13751: Add BooleanSimilarityFactory class.
(Andy Webb via Christine Poerschke)
- SOLR-14875: Make SolrEventListeners load from packages
(noble)
- SOLR-10391: ConfigSet handler now supports overrides on existing configsets.
(Tomás Fernández Löbbe)
- SOLR-11167: Avoid $SOLR_STOP_WAIT use during 'bin/solr start' if $SOLR_START_WAIT is supplied.
(Omar Abdelnabi, Christine Poerschke)
- SOLR-14907: Support single file upload/overwrite in configSet API.
(Houston Putman)
- SOLR-14914: Add option to disable metrics collection.
(ab)
- SOLR-14588: Introduce Circuit Breaker Infrastructure and a JVM heap usage memory tracking circuit breaker implementation
(Atri Sharma)
- SOLR-14615: CPU Utilization Based Circuit Breaker
(Atri Sharma)
- SOLR-15052: Reducing overseer bottlenecks using per-replica states
(noble, Ishan Chattopadhyaya)
- Improvements (13)
- SOLR-14523: Enhance gradle logging calls validation: eliminate getMessage()
(Andras Salamon via Erick Erickson)
- SOLR-14537: Improve performance of ExportWriter.
(ab, Joel Bernstein)
- SOLR-14566: Request ID's ('rid') are now added by default to distributed search requests, and can be used to correlate
logs from the receiving coordinator node with those from downstream shard requests. This can be disabled by providing a
disableRequestId=true request parameter.
(Jason Gerlowski)
- SOLR-14635: ThreadDumpHandler has been enhanced to show lock ownership
(hossman)
- SOLR-11262: Add support to stream data out using push APIs when wt=xml
(yonik, Munendra S N)
- SOLR-13205: Prevent StringIndexOutOfBoundsException when parsing field names in SolrQueryParserBase
(pramodkumar9 via Jason Gerlowski)
- SOLR-14651: The MetricsHistoryHandler can more completely disable itself when you tell it to.
Also, it now shuts down more thoroughly.
(David Smiley)
- SOLR-14722: Track timeAllowed from earlier in the request lifecycle.
(David Smiley)
- SOLR-13438: On deleting a collection, its config set will also be deleted iff it has been auto-created, and not used by any other collection
(Anderson Dorow)
- SOLR-14802: geodist: Support most (all?) spatial field types as an argument like LLPSF, SRPTFT, and others.
(Tom Edge, Craig Wrigglesworth)
- SOLR-14905: Update commons-io version to 2.8.0 due to security vulnerability.
(Nazerke Seidan via Bruno Roustant)
- SOLR-14691: Metrics reporting should avoid creating objects.
(ab, noble)
- SOLR-14933: Ability to add T and P type replica from admin UI
(Sayan Das via Eric Pugh)
- Optimizations (8)
- SOLR-14354: HttpShardHandler send requests in async
(Cao Manh Dat).
- SOLR-14610: ReflectMapWriter to use MethodHandle instead of old reflection
(noble)
- SOLR-13132: JSON Facet perf improvements to support "sweeping" collection of "relatedness()"
(hossman, Michael Gibney)
- SOLR-14422: Admin UI shows Angular placeholders on first load / refresh.
(Colvin Cowie via Eric Pugh)
- SOLR-14819: Fix inefficient iterator pattern in JsonSchemaValidator.
(Thomas DuBuisson via Bruno Roustant)
- SOLR-14658: SolrJ's CollectionAdminRequest.collectionStatus(collection) would internally get
all collection statuses instead of just the specified collection.
(Andy Vuong)
- SOLR-14576 : Do not use SolrCore as keys in a WeakHashMap
(noble)
- SOLR-14776: Precompute fingerprint during PeerSync in parallel with fetching fingerprint from replicas.
(Cao Manh Dat, Mike Drob via shalin)
- Bug Fixes (19)
- SOLR-14869: ChildDocTransformer should have omitted deleted child documents.
(Bar Rotstein, David Smiley)
- SOLR-11656: TLOG replication doesn't work properly after rebalancing leaders.
(Yuki Yano via
Erick Erickson)
- SOLR-14516: Fix NPE in JSON response writer(wt=json) with /get when writing non-stored, non-indexed docvalue field
from an uncommitted document
(noble, Ishan Chattopadhyaya, Munendra S N)
- SOLR-14657: Improve error handling in IndexReader realted metrics that were causing scary ERROR logging
if metrics were requested while Solr was in the process of closing/re-opening a new IndexReader.
(hossman)
- SOLR-14748: Fix incorrect auth/SSL startup logging
(Jason Gerlowski)
- SOLR-14751: Zookeeper Admin screen not working for old ZK versions
(janhoy)
- SOLR-14677: Improve DIH termination logic to close all DataSources, EntityProcessors
(Jason Gerlowski)
- SOLR-14703: Fix edismax replacement of all whitespace characters with spaces
(Yuriy Koval via Jason Gerlowski)
- SOLR-14700: Avoid NullPointerException in TupleStream.getShards() when streamContext is null.
(Mads Bondo Dydensborg, Christine Poerschke, Mike Drob)
- SOLR-14752: Fix error in Zookeeper status when Prometheus plugin is enabled in ZK
(Philipp Trulson via janhoy)
- SOLR-14774: HealthCheckHandler is no longer an implicit SolrCore handler and can be configured from solr.xml
(Tomás Fernándex Löbbe)
- SOLR-14714: Solr.cmd in windows loads the incorrect jetty module when using java>=9
(Endika Posadas via
Erick Erickson)
- SOLR-14821: {!terms} docValuesTermsFilterTopLevel method now works with single-valued strings
(Anatolii Siuniaev
via Jason Gerlowski)
- SOLR-13181: macro expansion of parameters could result in a StringIndexOutOfBoundsException.
Also, params with macros will be processed more strictly by LTR FeatureWeight.
(David Smiley, Cesar Rodriguez, Christine Poerschke)
- SOLR-14503: Use specified waitForZk value as connection timeout for zookeeper in SolrDispatcherFilter.
Also, consume specified SOLR_WAIT_FOR_ZK in bin/solr.cmd
(Colvin Cowie via Munendra S N)
- SOLR-14936: Fixed Grafana dashboard filters for collection, shard, replica and core
(janhoy)
- SOLR-14924: Some ReplicationHandler metrics are reported using incorrect types.
(ab)
- SOLR-14483: Fix empty drop down for adding replica in Admin UI
(Sayan Das via Eric Pugh)
- SOLR-14549: Fix listing of Files in a Directory on Solr Admin UI
(Kevin Risden)
- Other Changes (15)
- SOLR-14592: Upgrade Zookeeper to 3.6.1. NOTE: this required upgrading netty to 4.1.50
(Erick Erickson)
- SOLR-10742: SolrCores.getNamesForCore is quite inefficient and blocks other core operations.
NOTE: this experimental method has been removed
(Erick Erickson)
- SOLR-13939: Extract any non-gradle related patches (deprecations, URL fixes, etc.) from gradle effort. NOTE:
this will be in several separate commits/pushes.
(Erick Erickson)
- SOLR-14637: Update CloudSolrClient examples to remove deprecated method.
(Andras Salamon via Eric Pugh)
- SOLR-14676: Update commons-collections to 4.4 and use it in Solr
(Erick Erickson)
- SOLR-11868: Deprecate CloudSolrClient.setIdField, use information from Zookeeper
(Erick Erickson)
- SOLR-14641: PeerSync, remove canHandleVersionRanges check
(Cao Manh Dat)
- SOLR-14731: Rename @SolrSingleThreaded to @SolrThreadUnsafe, mark DistribPackageStore with the annotation
(marcussorealheis)
- SOLR-14579: Comment SolrJ 'Utils' generic map functions
(Megan Carey and a lot of education from Uwe Schindler via Erick Erickson)
- SOLR-10471: Update default Zookeeper session timeout to 30s in bin/solr and bin/solr.cmd
(Michael Braun, Markus Jelsma via Munendra S N)
- SOLR-14876: Upgrade to zookeeper 3.6.2
(odidev via Erick Erickson)
- SOLR-14333: Implement toString in Collapse filter so that proper parsed queries returned in debug response.
Also, Deprecate unused constants NULL_COLLAPSE, NULL_IGNORE, NULL_EXPAND, HINT_MULTI_DOCVALUES in collapse parser.
(Guna Sekhar Dora Kovvuru, Munendra S N, Mike Drob)
- SOLR-12987: Deprecated plugins/features are now logged once and with log category org.apache.solr.DEPRECATED
(David Smiley)
- SOLR-14910: Use in-line tags for logger declarations in Gradle ValidateLogCalls that are non-standard,
change //logok to //nowarn
(Erick Erickson)
- SOLR-14659: Remove restlet as dependency for the ManagedResource API
(Timothy Potter, noble)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Bug Fixes (5)
- SOLR-14859: DateRangeField now throws errors when invalid field/fieldType options specified; no longer silently
accepts incompatible option values
(Jason Gerlowski, Chris Hostetter, Munendra S N)
- SOLR-14768: Fix HTTP multipart POST to Solr -- a regression from 8.6.0. Many Jetty classes are not classpath-visible
from the Solr webapp.
(David Smiley)
- SOLR-14850: Fix ExactStatsCache NullPointerException when shards.tolerant=true.
(Yevhen Tienkaiev via ab)
- SOLR-14897: Fix unlimited number of forwarding the request from one node to another node.
(hossman, Munendra S N)
- SOLR-14898: Stop returning duplicate HTTP response headers when requests are forward to another node.
(hossman)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Bug Fixes (2)
- SOLR-14751: Zookeeper Admin screen not working for old ZK versions
(janhoy)
- SOLR-14663: Copy ConfigSet root data from base ConfigSet when using CREATE command
(Andras Salamon, Tomás Fernández Löbbe)
- Bug Fixes (2)
- SOLR-14665, SOLR-14706: Revert SOLR-12845 adding of default autoscaling cluster policy, due to performance
issues.
(Ishan Chattopadhyaya, Houston Putman, Gus Heck, ab)
- SOLR-14671: Parsing dynamic ZK config sometimes cause NuberFormatException
(janhoy)
- Consult the lucene/CHANGES.txt file for additional, low level, changes in this release.
- New Features (12)
- SOLR-14210: HealthCheckHandler can now require that all cores are healthy before returning 'OK'
To enable, add &requireHealthyCores=true to enable
(janhoy, Houston Putman, shalin)
- SOLR-13942: A read API at /api/cluster/zk/* to fetch raw ZK data and view contents of a ZK directory
(noble)
- SOLR-14237: A new panel with security info in admin UI's dashboard
(Ishan Chattopadhyaya, Moshe Bla)
- SOLR-12131: ExternalRoleRuleBasedAuthorizationPlugin which gets user's roles from request
(janhoy)
- SOLR-14478: Allow the diff Stream Evaluator to operate on the rows of a matrix
(Joel Bernstein)
- SOLR-14476: Add percentiles and standard deviation aggregations to stats, facet and
timeseries Streaming Expressions
(Joel Bernstein)
- SOLR-14470: Add streaming expressions to /export handler.
(ab, Joel Bernstein)
- SOLR-14404: CoreContainer level custom requesthandlers
(noble)
- SOLR-13749: Join queries can now work cross-collection, even when sharded or when spanning nodes.
(Dan Fox, Kevin Watters, via Gus Heck, David Smiley)
- SOLR-14481: Add drill Streaming Expression for efficient and accurate high cardinality aggregation.
(Joel Bernstein)
- SOLR-14599: Package manager support for cluster level plugins (see SOLR-14404)
(Ishan Chattopadhyaya)
- SOLR-14590: Add support for RankFields. RankFields allow the use of per-document scoring factors in a
way that lets Solr skip over non-competitive documents when ranking. See SOLR-13289.
(Tomás Fernández Löbbe, Varun Thacker)
- Improvements (19)
- SOLR-14316: Remove unchecked type conversion warning in JavaBinCodec's readMapEntry's equals() method
(Aroop Ganguly, Tomás Fernández Löbbe, Noble Paul, Anshum Gupta)
- SOLR-14260: Make SolrJ ConnectionSocketFactory pluggable via SocketFactoryRegistryProvider setting on HttpClientUtil
(Andy Throgmorton via David Smiley)
- SOLR-14342: Load cores in an order that makes collections available sooner and reduces leaderVoteWait timeouts in
large SolrCloud clusters.
(David Smiley)
- SOLR-14329: Add support to choose collapse group to expand in ExpandComponent based on cost
(Munendra S N)
- SOLR-14307: User defined "<cache/>" entries in solrconfig.xml now support enabled="true|false" just like
core searcher caches.
(hossman)
- SOLR-14364: LTR's SolrFeature "fq" now supports PostFilters (e.g. collapse).
(David Smiley)
- SOLR-14396: TaggerRequestHandler should not error on an empty index
(Trey Grainger)
- SOLR-12845: Add a default autoscaling cluster policy.
(shalin, varunthacker, ab)
- SOLR-14433: Metrics: SolrShardReporter's default metrics list now includes TLOG and UPDATE./update
(David Smiley)
- SOLR-14423: Move static SolrClientCache from StreamHandler to CoreContainer for wider reuse and better life-cycle management.
(ab)
- SOLR-14407: Handle shards.purpose in the postlogs tool
(Joel Bernstein)
- SOLR-13325: ComputePlanAction now supports a collection selector of the form `collections: {policy: my_policy}`
which can be used to select multiple collections that match collection property/value pairs. This is useful to
maintain a whitelist of collections for which actions are taken without needing to hard code the collection names
themselves. The collection hints are pushed down to the policy engine so operations for non-matching collections
are not computed at all.
(ab, shalin)
- SOLR-14419: json.queries as well as other parameters might be referred via {"param":"ref"} in Query DSL
(Mikhail Khludnev)
- SOLR-11334: hl.fl and tv.fl now parse field lists when they have both commas and spaces
(David Smiley, Yasufumi Mizoguchi)
- SOLR-14442: bin/solr and bin\solr.cmd invoke jstack <SOLR_PID> before forceful termination, if jstack is available.
Also, bin\solr.cmd executes forceful termination even port is unbinded already
(Christine Poerschke via Mikhail Khludnev).
- SOLR-14384: SolrRequestInfo now stacks internally when a new request is set/clear'ed.
Also fixes SolrIndexSearcher.warm which should have re-instated previous SRI.
(Nazerke Seidan, David Smiley)
- SOLR-14561: CoreAdminAPI's parameters instanceDir and dataDir are now validated, and must be relative to either
SOLR_HOME, SOLR_DATA_HOME or coreRootDir. Added new solr.xml config 'allowPaths', controlled by system property
'solr.allowPaths' that allows you to add other allowed paths when needed.
- SOLR-13286: Metrics will no longer write a (sometimes large) log message every minute. This can be re-enabled
via log4j2.xml if desired, or other HttpSolrCall log messages may be quieted on a per handler basis.
(Gus Heck)
- SOLR-14539: Introducing {!bool excludeTags=...} for Query DSL.
(Mikhail Khludnev)
- Optimizations (10)
- SOLR-8306: Do not collect expand documents when expand.rows=0
(Marshall Sanders, Amelia Henderson)
- SOLR-14340: Remove unnecessary configset verification checks when SolrCloud merely wants to know which configset a
collection is using. Improves CLUSTERSTATUS times for massive clusters.
(Mathieu Marie, David Smiley)
- SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values.
(Cao Manh Dat)
- SOLR-14376: Optimize filter queries that match all docs.
(David Smiley)
- LUCENE-7788: fail precommit on unparameterised log messages and examine for wasted work/objects
(Erick Erickson)
- SOLR-13289: When the "minExactCount" parameters is provided in queries and it's value is lower than the number of hits,
Solr can speedup the query resolution by using the Block-Max WAND algorithm (see LUCENE-8135). When doing this, the
value of matching documents in the response (numFound) will be an approximation.
(Ishan Chattopadhyaya, Munendra S N, Tomás Fernández Löbbe, David Smiley)
- SOLR-14472: Autoscaling "cores" preference now retrieves the core count more efficiently, and counts all cores.
(David Smiley)
- SOLR-14552: Add BlockMax-WAND support to ReRank queries
(Tomás Fernández Löbbe)
- SOLR-14554: Add BlockMax-WAND support for queries where the score is requested
(Tomás Fernández Löbbe)
- SOLR-14634: Limit the HTTP security headers to "/solr" end point
(noble)
- Bug Fixes (33)
- SOLR-13264: IndexSizeTrigger aboveOp / belowOp properties not in valid properties.
(Christine Poerschke, ab)
- SOLR-13199: Return proper error when invalid parentFilter is passed in ChildDocTransformer
(Johannes Kloos, Munendra S N, David Smiley, Mikhail Khludnev)
- SOLR-14347: Autoscaling placement wrong when concurrent replica placements are calculated.
(ab)
- SOLR-14128: Improve distributed locking around managed schema upgrade process.
(ab)
- SOLR-14302: Ensure Solr always includes the stacktrace for exceptions by using '-OmitStackTraceInFastThrow'
(hossman)
- SOLR-14317: HttpClusterStateProvider throws exception when only one node down.
(Lyle Wang via Ishan Chattopadhyaya)
- SOLR-14356: PeerSync should not fail with SocketTimeoutException from hanging nodes
(Cao Manh Dat)
- SOLR-14402: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search.
This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not
distributed search.
(shalin)
- SOLR-14291: Handle dotted fields in legacy Analytics Component
(Anatolii Siuniaev via Mikhail Khludnev)
- SOLR-14371: Zk StatusHandler now parse dynamic zk server config if supported, fixing Admin UI Zookeeper Status
screen in case dynamic reconfig host list differs from static zkHost string
(janhoy)
- SOLR-14421: New examples in solr.in.cmd in Solr 8.5 don't work as provided
(Colvin Cowie via janhoy)
- SOLR-14431: SegmentsInfoRequestHandler does not release IndexWriter
(Tiziano Degaetano, ab)
- SOLR-14463: Solr Admin ZkStatus page now works with ZK 3.6, without 'For input string: "null"' error
(janhoy, Bernd Wahlen)
- SOLR-14456: Fix Content-Type header usage when a request is forwarded from Solr node to Solr
node with compression enabled
(samuelgmartinez via Houston Putman)
- SOLR-8394: /admin/luke was always showing 0 for indexHeapUsageBytes. It should work now.
(Steve Molloy, Isabelle Giguere, David Smiley)
- SOLR-14492: Fix ArrayIndexOutOfBoundsException in json.facet 'terms' when FacetFieldProcessorByHashDV is
used with aggregations over multivalued numeric fields
(hossman)
- SOLR-14477: Fix incorrect 'relatedness()' calculations in json.facet 'terms' when 'prefix' option is used
(hossman)
- SOLR-14504: ZkController LiveNodesListener has NullPointerException in startup race.
(Colvin Cowie via ab)
- SOLR-14498: Upgrade to Caffeine 2.8.4, which fixes the cache poisoning issue.
(Jakub Zytka, ab)
- SOLR-14517: Dont ignore 'mm' localparam on edismax queries using operators
(Yuriy Koval via Jason Gerlowski)
- SOLR-14491: Intercepting internode requests in KerberosPlugin when HTTP/2 client is used
(Ishan Chattopadhyaya, Moshe Bla)
- SOLR-14525: SolrCoreAware, ResourceLoaderAware should be honored for plugin loaded from packages
(noble)
- SOLR-9679: When removing zk node /security.json, security is now disabled gracefully
(janhoy)
- SOLR-14520: Fixed server errors from the json.facet allBuckets:true option when combined with refine:true
(Michael Gibney, hossman)
- SOLR-14467: Fix relatedness() stat in json.facets to no longer cause server errors (or nonsense results)
when combined with allBuckets:true.
(Michael Gibney, hossman)
- SOLR-13203: Return 400 status code on invalid dynamic field for Edismax's user Fields
(Johannes Kloos, mrsoong via Munendra S N)
- SOLR-14550: Fix duplicates issue in Atomic updates with add-distinct
(Thomas Corthals, Munendra S N)
- SOLR-14345: Return proper error message when non-BinaryResponseParser is used in solrJ
(Munendra S N)
- SOLR-14516: NPE in JsonTextWriter
(noble)
- SOLR-14577: Return 400 BAD REQUEST when field is missing on a Terms query parser request
(Tomás Fernández Löbbe)
- SOLR-14409: Existing violations allow bypassing policy rules when adding new replicas
(noble, ab)
- SOLR-14584: Correct SOLR_SSL_KEY_STORE and SOLR_SSL_TRUST_STORE example comments in solr.in.sh and solr.in.cmd files
(Aren Cambre via Christine Poerschke)
- SOLR-14628: hl.fragsizeIsMinimum now defaults to true, thus fragments size like previous to Solr 8.5 and addresses a
significant performance regression.
(David Smiley, Michal Hlavac)
- Other Changes (56)
- SOLR-14197: SolrResourceLoader: marked many methods as deprecated, and in some cases rerouted exiting logic to avoid
them.
(David Smiley)
- SOLR-14312: Upgrade Zookeeper to 3.5.7
(Erick Erickson)
- SOLR-14296: Update netty to 4.1.46
(Andras Solaman via Erick Erickson)
- SOLR-10157: Improve error messages when unknown aggregations are specified in the request
(hossman, Munendra S N)
- SOLR-13944: Remove redundant checks in SpellCheckCollator and add tests for spellcheck with collapse
(Stefan, Munendra S N, Tomas Eduardo Fernandez Lobbe)
- SOLR-12353: SolrDispatchFilter expensive non-conditional debug line degrades performance
(Pascal Proulx via Erick Erickson)
- SOLR-13842: Remove redundant defaults from ImplicitPlugins.json
(Munendra S N)
- SOLR-13893: BlobRepository reads max jar size from `runtime.lib.size` system property. Old `runtme.lib.size` is Deprecated
(Erick Erickson, Kesharee Nandan Vishwakarma, Munendra S N)
- SOLR-14367: Upgrade Tika to 1.24
(Mibo via Erick Erickson)
- SOLR-14351: Hardened MDC logging; sometimes info was absent
(David Smiley)
- SOLR-14386: Update Jetty to 9.4.27 and dropwizard-metrics version to 4.1.5
(Erick Erickson)
- SOLR-9909: SolrjNamedThreadFactory is deprecated in favor of SolrNamedThreadFactory. DefaultSolrThreadFactory is
removed from solr-core in favor of SolrNamedThreadFactory in solrj package and all solr-core classes now use
SolrNamedThreadFactory.
(Andras Salamon, shalin)
- SOLR-14391: Removed internal-ish ScoreFilter marker interface; only used by {!collapse}. Removed needless code in
SolrIndexSearcher.getDocSet(List<Query>) (processes some filter queries).
(David Smiley)
- SOLR-13886: HDFSSyncSliceTest and SyncSliceTest started failing frequently
(Kevin Risden)
- SOLR-14173: Major redesign of the Solr Reference Guide
(Cassandra Targett)
- SOLR-14461: Replaced commons-fileupload dependency with using Jetty's facilities.
(David Smiley)
- SOLR-14466: Upgrade log4j2 to latest release (2.13.2)
(Erick Erickson)
- SOLR-11934: Visit Solr logging, it's too noisy. Note particularly that the messages for
opening a new searcher have changed and include the autowarm time.
(Erick Erickson)
- SOLR-7880: Update commons-cli to 1.4
(Erick Erickson)
- SOLR-14226: Fix or suppress 14 resource leak warnings in apache/solr/core
(Andras Salaman via
Erick Erickson)
- SOLR-14482: Fix or suppress warnings in solr/search/facet
(Erick Erickson)
- SOLR-14485: Fix or suppress 11 resource leak warnings in apache/solr/cloud
(Andras Salaman via
Erick Erickson)
- SOLR-14495: Fix or suppress warnings in solr/search/function
(Erick Erickson)
- SOLR-14280: Improve error reporting in SolrConfig
(Andras Salamon via Jason Gerlowski)
- SOLR-14474: Fix remaining auxilliary class warnings in Solr
(Erick Erickson)
- SOLR-14519: Fix or suppress warnings in solr/cloud/autoscaling/
(Erick Erickson)
- SOLR-14526: fix or suppress warnings in apache/solr/core
(Erick Erickson)
- SOLR-14533: Fix or suppress warnings in solr/handler/admin
(Andras Salamon, Erick Erickson)
- SOLR-14535: Fix or suppress warnings in apache/solr/handler/component
(Erick Erickson)
- SOLR-14538: Fix or suppress remaining warnings in apache/solr/handler
(Erick Erickson)
- SOLR-14536: Fix or suppress warnings in apache/solr/common
(Erick Erickson)
- SOLR-14480: Fix or suppress warnings in solr/cloud/api
(Erick Erickson)
- SOLR-13492: Ensure explicit GCs are concurrent by adding '+ExplicitGCInvokesConcurrent'.
(Guna Sekhar Dora, Shawn Heisey, Munendra S N)
- SOLR-14542: Fix or suppress warnings in solr/handler/dataimport
(Erick Erickson)
- SOLR-14544: Fix or suppress warnings in solr/client/solrj/io/eval
(Erick Erickson)
- SOLR-14543: Fix or suppress warnings in apache/solr/search
(Erick Erickson)
- SOLR-14545: Fix or suppress warnings in apache/solr/update
(Erick Erickson)
- SOLR-14548: Address warning: static member should be qualified by type name
(Mike Drob)
- SOLR-14547: Fix or suppress warnings in solr/client/solrj/io/stream
(Erick Erickson)
- SOLR-14455: Fix or suppress warnings in solr/test-framework
(Erick Erickson)
- SOLR-14559: Fix or suppress warnings in solr/core/src/java/org/apache/solr/util,
response, cloud, security, schema, api
(Erick Erickson)
- SOLR-14563: Fix or suppress warnings in solr/contrib
(Erick Erickson)
- SOLR-14565: Fix or suppress warnings in solrj/impl and solrj/io/graph
(Erick Erickson)
- SOLR-14564: Fix or suppress remaining warnings in solr/core
(Erick Erickson)
- SOLR-14567: Fix or suppress remaining warnings in solrj
(Erick Erickson)
- SOLR-14556: Fix or suppress warnings in solrj/cloud/autoscaling
(Erick Erickson)
- SOLR-14573: Fix or suppress warnings in solrj/src/test
(Erick Erickson)
- SOLR-14574: Fix or suppress warnings in solr/core/src/test (parts 1 and 2)
(Erick Erickson)
- SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(Erick Erickson)
- LUCENE-9411: Fail complation on warnings, 9x gradle-only (Erick Erickson, Dawid Weiss)
Only mentioned in 8.6 because I backported some more warning suppressions but not
the fail-on-warnings.
- SOLR-14589: Remove IntelliJ //noinspection comments
(Erick Erickson)
- SOLR-14541: Ensure classes that implement equals implement hashCode or suppress warnings
(gezapeti, Ilan Ginsburg, Erick Erickson)
- SOLR-14022: Deprecate CDCR
(Joel Bernstein, Ishan Chattopadhyaya)
- SOLR-14066: Data Import Handler is deprecated. It is scheduled to be removed as of 9.0 and a community supported
package for the same may now be used instead.
(Ishan Chattopadhyaya, janhoy)
- SOLR-14021: HDFS support is deprecated. It is scheduled to be removed as of 9.0. Please see JIRA for more details.
(Joel Bernstein, Ishan Chattopadhyaya)
- SOLR-14603: Upgrade Restlet to 2.4.3
(marcussorealheis)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Bug Fixes (2)
- SOLR-14411: Fix Admin UI collection/core drop-downs placeholder text. Completes work started in SOLR-14359
(janhoy)
- SOLR-14471: Fix bug in shards.preference behavior, base replica selection strategy not applied to the last group of
equivalent replicas.
(Michael Gibney via Tomás Fernández Löbbe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Bug Fixes (1)
- SOLR-14359: Admin UI collection/core drop-downs had wrong placeholder text. Fixed by upgrading angular-chosen
This fix was incomplete, see SOLR-14411 for the followup
(janhoy)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- New Features (7)
- SOLR-12490: Introducing json.queries in JSON Request API. Every property of this object holds one or many named
Query DSL queries. It's optional and doesn't impact response without explicit referencing these queries by names
(Anatolii Siuniaev via Mikhail Khludnev)
- SOLR-14130: Add postlogs command line tool for indexing Solr logs
(Joel Bernstein)
- SOLR-13749: New cross collection join filter (XCJF)
(Dan Fox, Kevin Watters, via Gus Heck)
- SOLR-12325: Introducing uniqueBlock({!v=type:parent}) aggregation
(Anatolii Siuniaev via Mikhail Khludnev)
- SOLR-14241: New delete() Stream Decorator
(hossman)
- SOLR-14139: Support backtick phrase queries in Streaming Expressions
(Joel Bernstein)
- SOLR-12238: Synonyms can be individually boosted by marking them with '|<boost>' in
the synonyms file, and adding a DelimitedBoostTokenFilter to the analysis chain
(Alessandro Benedetti, Alan Woodward)
- Improvements (22)
- SOLR-14120: Define JavaScript methods 'includes' and 'startsWith' to ensure AdminUI can be displayed when using
Internet Explorer 11
(jafurrer).
- SOLR-14042: Fix varargs precommit warnings
(Andraas Salamon via Jason Gerlowski)
- SOLR-14095: Replace Java serialization with Javabin in the Overseer queues
(Tomás Fernández Löbbe)
- SOLR-14125: Make <expressible> plugins work with packages
(noble)
- SOLR-14138: Jetty's NCSA-formatted RequestLog can be enabled via environment variable,
SOLR_REQUESTLOG_ENABLED=true. This will respect SOLR_LOGS_DIR if it is set.
(rmuir)
- SOLR-14136: Jetty's InetAccessHandler can be enabled via environment variables,
SOLR_IP_WHITELIST and SOLR_IP_BLACKLIST. These variables can restrict access to
Solr based on IP addresses/networks.
(rmuir)
- SOLR-14131: Add maxQueryLength option to DirectSolrSpellchecker.
(Andy Webb via Bruno Roustant)
- SOLR-13984: Java's SecurityManager sandbox can be enabled via environment variable,
SOLR_SECURITY_MANAGER_ENABLED=true.
(rmuir)
- SOLR-14129: Reuse Jackson ObjectMapper in AuditLoggerPlugin
(janhoy)
- LUCENE-9093: The Unified highlighter has two new passage sizing parameters, hl.fragAlignRatio and
hl.fragsizeIsMinimum, with defaults that aim to better center matches in fragments than previously. See the ref guide.
Regardless of the settings, the passages may be sized differently than before.
(Nándor Mátravölgyi, David Smiley)
- SOLR-14154: Return correct isolation level when retrieving it from the SQL Connection
(Nick Vercammen, Kevin Risden)
- SOLR-14132: Upgrade Angular JS 1.3.8 to 1.7.9
(Kevin Risden)
- SOLR-14221: Upgrade restlet to version 2.4.0
(janhoy)
- SOLR-10567: Add Support for DateRangeField in JSON Facet range
(Stephen Weiss, Munendra S N)
- SOLR-14245: Validate Replica / ReplicaInfo on creation.
(ab)
- SOLR-14194: Highlighting now works when the uniqueKey field is not stored but has docValues. And the original
highlighter can now highlight text fields from docValues.
(Andrzej Wislowski, David Smiley)
- SOLR-14270: export command to have an option to write to a zip file
(noble)
- SOLR-14286: Upgrade Jaegar to 1.1.0
(Cao Manh Dat)
- SOLR-14281: Make sharedLib configurable through system property. Also allow multiple paths, separated by comma.
See Reference Guide 'format-of-solr-xml.adoc' and comment in 'solr.in.sh' for details
(janhoy)
- SOLR-13892: New "top-level" docValues join implementation
(Jason Gerlowski, Joel Bernstein)
- SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or shapes.
(Adrien Grand)
- SOLR-13965: In GraphHandler, support <expressible> configuration and deprecate <streamFunctions> configuration.
(Eric Pugh via Christine Poerschke)
- Optimizations (3)
- SOLR-13890: Add "top-level" DV implementation for {!terms} queries. This approach tends to be more efficient for
queries with very large numbers of terms. The new implementation is used by default for method=docValuesTermsFilter
terms queries that are searching 500 or more terms. Users wishing to ignore this heuristic can choose the
docValuesTermsFilterTopLevel or docValuesTermsFilterPerSegment methods directly.
(Jason Gerlowski, Joel Bernstein, David Smiley)
- SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query DSL are cached by default
(Mikhail Khludnev)
- SOLR-14040: The shareSchema solr.xml option now supports SolrCloud. shareSchema is useful when you have many
cores per node and large schemas.
(David Smiley)
- Bug Fixes (29)
- SOLR-14099: Fixed @LogLevel annotation in test-framework to correctly 'unset' Loggers after test
(hossman)
- SOLR-14017: Ensure bin/solr's -q and -v options work with examples
(Andy Vuong via Jason Gerlowski)
- SOLR-14122: SimUtils converts v2 to v1 request params incorrectly.
(Li Cao, ab)
- SOLR-13089: Fix lsof edge cases in the solr CLI script
(Martijn Koster via janhoy)
- SOLR-14198: Nullpointer exception in AuditEvent when initialized from AuthorizationContext
(janhoy)
- SOLR-6613: TextField.analyzeMultiTerm does not throw an exception when Analyzer returns no terms.
(Bruno Roustant)
- SOLR-14192: Race condition between SchemaManager and ZkIndexSchemaReader.
(ab)
- SOLR-12859: Fixed DocExpirationUpdateProcessorFactory to work with BasicAuth and other auth plugins
that delegate to PKI for server initiated node-to-node communication.
(hossman)
- SOLR-11746: Adding existence queries for PointFields. DocValuesFieldExistsQuery and NormsFieldExistsQuery are used for existence queries when possible.
(Houston Putman, hossman, Kai Chan)
- SOLR-14207: Fix logging statements with less or more arguments than placeholders.
(shalin)
- SOLR-14172: Collection metadata remains in zookeeper if too many shards are requested. This also fixes a bug where
an inability to assign a node based on existing autoscaling policy resulted in a server error instead of a bad request.
(Andras Salamon, Kevin Risden, shalin)
- SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown=false
(janhoy)
- SOLR-14205: Do not fail when given timeout to connectionImpl.isValid() = 0
(Nick Vercammen via Kevin Risden)
- SOLR-11554: Support handling OPTIONS request for Hadoop authentication filter
(Peter Cseh via Kevin Risden)
- SOLR-14189: Switch from String.trim() to StringUtils.isBlank() in query parsers
(Andy Webb via Uwe Schindler)
- SOLR-13897: Fix unsafe publication of Terms object in ZkShardTerms that can cause visibility issues
and race conditions under contention.
(shalin)
- SOLR-14090: Handle the case in `delete-copy-field` when source is a dynamic field
(Frank Iversen, Munendra S N)
- SOLR-14219: SOLR-14095 Introduced an issue for rolling restarts (Incompatible Java serialization). This change
Fixes the compatibility issue while keeping the functionality in SOLR-14095.
(Andy Webb, Tomás Fernández Löbbe)
- SOLR-13887: Use default idleTimeout instead of 0 for HTTP2 requests.
(Houston Putman)
- SOLR-14038: Admin UI display for "state.json" should be in a scollable region
(Kevin Risden)
- SOLR-14058: Fix IndexOutOfBoundsException in PeerSync that can prevent nodes from recovering
under certain circumstances.
(yonik)
- SOLR-13041: Add hashCode for autoscaling.Condition to accompany the already present equals.
(Zsolt Gyulavari via Christine Poerschke)
- SOLR-12550: ConcurrentUpdateSolrClient doesn't respect timeouts for commits and optimize.
(Marc A. Morissette, Bérénice MAUREL via shalin)
- SOLR-14250: Do not log error when trying to consume non-existing input stream due to Expect: 100-continue
(janhoy)
- SOLR-13411: Deny atomic update for route.field, uniqueKey, version and throw exception.
(Dr Oleg Savrasov via Mikhail Khludnev)
- SOLR-14252: Avoid NullPointerException in AggregateMetric.
(Andy Webb via ab)
- SOLR-14287: Admin UI Properties screen does not show colons
(janhoy)
- SOLR-14299: IndexFetcher doesn't reset error count to 0 after the last packet is received
(Pushkar Raste via Mike Drob)
- SOLR-14073: Fix segment look ahead NPE in CollapsingQParserPlugin
(Joel Bernstein)
- Other Changes (16)
- SOLR-14047: Make sure tests don't pickup other Hadoop installs
(Kevin Risden)
- SOLR-14054: Upgrade to Tika 1.23 when available
(Tim Allison)
- SOLR-14091: Remove deprecated soLingerTime when configuring Jetty connector
(Matthias Krueger via Kevin Risden)
- SOLR-13778: Solrj client will retry requests on SSLException with a suppressed SocketException
(very likely a hard-closed socket connection).
- SOLR-14169: Fix 20 Resource Leak warnings in SolrJ's apache/solr/common
(Andras Salamon via Tomás Fernández Löbbe)
- SOLR-14184: Internal 'test' variable DirectUpdateHandler2.commitOnClose has been removed and replaced with
TestInjection.skipIndexWriterCommitOnClose. Users that modified DUH2.commitOnClose in test cases for custom
plugins/modicitations should now use TestInjection instead.
(hossman)
- SOLR-14248: Improve ClusterStateMockUtil and make its methods public.
(shalin)
- SOLR-14209: Upgrade JQuery to 3.4.1
(Kevin Risden)
- SOLR-13996: Refactor HttpShardHandler.prepDistributed method.
(shalin)
- SOLR-13794: Replace redundent test only copy of '_default' configset with SolrTestCase logic to correctly
set 'solr.default.confdir' system property
(hossman)
- SOLR-14263: Update jvm-settings.adoc
(Erick Erickson)
- SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches
during SolrCore reload. Removed Bandaid kludge.
(Erick Erickson)
- SOLR-10306: Document in Reference Guide how to disable or reduce swapping
(janhoy)
- SOLR-13983: Process execution is removed from SystemInfoHandler. A best-effort attempt to
- execute `uname -a` and "`uptime`" on non-Windows platforms is no longer made.
(rmuir)
- SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in production
(janhoy)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.5.5
- Jetty 9.4.19.v20190610
- Upgrade Notes (1)
- SOLR-14158: Package manager now stores keys in package store instead of ZK. If you're using package manager and have already
added any repositories, then you would need to re-add all the keys from /keys/exe location, one-by-one, using:
"bin/solr add-key <keyfile.der>"
- Bug Fixes (4)
- SOLR-14106: Cleanup Jetty SslContextFactory usage
(Ryan Rockenbaugh, Jan Hoydahl, Kevin Risden)
- SOLR-14109: Always log to stdout from server/scripts/cloud-scripts/zkcli.{bat|sh}
(janhoy)
- SOLR-14165: SolrResponse serialVersionUID has changed in a backward incompatible way
(Andy Webb via noble)
- SOLR-14163: SOLR_SSL_CLIENT_HOSTNAME_VERIFICATION needs to work with Jetty server/client SSL contexts
(Kevin Risden)
- Improvements (1)
- SOLR-14158: Package manager to store public keys in a special "trusted" location instead of in ZooKeeper
(noble, Ishan Chattopadhyaya)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.5.5
- Jetty 9.4.19.v20190610
- Upgrade Notes (11)
- SOLR-13823: org.apache.solr.search.grouping.distributed.command.QueryCommand.Builder has new method 'setMainQuery' which is used
to set top-level query. build() would fail if called without setting mainQuery.
- SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to transition their
configurations to use org.apache.solr.search.CaffeineCache instead.
(ab)
- SOLR-13982: Some security-related http headers such as Content-Security-Policy are now set. If you have custom html served
up by Solr's http server that contains inline javascript, it will no longer execute in modern browsers. You can fix your JS
code to not run inline anymore, or edit etc/jetty.xml and weaken the CSP, or remove/alter the headers with a reverse proxy.
(rmuir)
- SOLR-13806: QueryResponse.getExplainMap() type has changed from Map<String, String> to Map<String, Object>
in order to accommodate structured explanations. This is a partially back-compatible change - compiled
third-party components will work the same as before due to type erasure but source code changes may be
required.
- SOLR-14025: VelocityResponseWriter has been hardened - only trusted configsets can render configset provided
templates and rendering templates from request parameters has been removed.
- SOLR-13904: timeAllowed parameter is allowed to have 0 value
(Houston Putman, Mikhail Khludnev)
- SOLR-13970: Using Collapse filter or expand component with grouping is explicitly disallowed as the combination
would cause inconsistent behavior and NPEs.
- SOLR-13978: The default configset no longer has the following:
-
Library inclusions (<lib ../>) for extraction, solr-cell libs, clustering, velocity, LTR and language identifier
-
/browse, /tvrh and /update/extract handlers
-
TermVector component
(if someone wants it, can be added using config APIs)
-
XSLT response writer
-
Velocity response writer
If you want to use them in your collections, please add them to your configset manually or through the Config APIs.
- SOLR-14071: Untrusted configsets (ones that are uploaded via unsecured configset API) cannot use <lib> directive.
Consider enabling authentication/authorization so that the uploaded configsets are trusted.
Note: If you already have a collection using untrusted configset that uses <lib> directive, it will not load after
upgrading to 8.4. You can re-upload your configset using "bin/solr zk -upconfig .." or place your libraries in the
classpath and restart Solr.
- SOLR-14065: VelocityResponseWriter has been deprecated and may be removed in a future version.
- SOLR-14072: The "Blob Store" API and "runtimeLib" plugin mechanism that uses it is now considered deprecated. The
replacement to it is the "Package Management" system, which includes a "File Store". These are experimental currently
but will grow/stabalize/mature.
- New Features (9)
- SOLR-13821: A Package store to store and load package artifacts
(noble, Ishan Chattopadhyaya)
- SOLR-13822: A Package management system with the following features:
(a) A packages.json in ZK to store the configuration,
(b) APIs to read/edit them, and
(c) Isolated classloaders to load the classes from those packages when the 'class'
attribute is prefixed with '<package-name>:'
(noble, Ishan Chattopadhyaya)
- SOLR-13662: A CLI based Package Manager ("bin/solr package help" for more details).
(Ishan Chattopadhyaya, noble, David Smiley, Jan Hoydahl)
- SOLR-10786: Add DBSCAN clustering Streaming Evaluator
(Joel Bernstein)
- SOLR-13911: Add 'missing' aggregation in JSON FacetModule
(hossman, Munendra S N)
- SOLR-13912: Add 'countvals' aggregation in JSON FacetModule
(hossman, Munendra S N)
- SOLR-12217: Support shards.preference in SolrJ for single shard collections. The parameter
is now used by the CloudSolrClient and Streaming Expressions.
(Houston Putman, Tomas Fernandez-Lobbe)
- SOLR-14043: Allow the precision Stream Evaluator to operate on matrices
(Joel bernstein)
- SOLR-14060: Allow the random Streaming Expression to create the x-axis
(Joel Bernstein)
- Improvements (16)
- SOLR-13731: 'javabin' must support a 1:1 mapping of the JSON update format
(noble)
- SOLR-13831: Support defining arbitrary autoscaling simulation scenarios.
(ab)
- SOLR-13865: Move replica routing code to SolrJ.
(Houston Putman via Tomas Fernandez-Lobbe)
- SOLR-13844: Remove replica recovery terms with the replica term
(Houston Putman via Cao Manh Dat)
- SOLR-13898: Use the new SolrCache.computeIfAbsent instead of non-atomic get / put in order to
improve cache hit ratios.
(ab, Ben Manes)
- SOLR-13907: In the Cloud/tree section of the Admin UI, fix metadata panel and force an horizontal scrollbar for the tree
(Richard Goodman via Tomás Fernández Löbbe)
- SOLR-13905: Make findRequestType in AuditEvent more robust
(janhoy)
- SOLR-13961: When using partial/atomic updates to remove child documents, we now support setting to null or
an empty list as equivalent to the "remove" command.
(Thomas Wöckinger)
- SOLR-13969: Clean up and document AuditEvent API
(janhoy)
- SOLR-13968: Support postingsFormat and docValuesFormat in schema fields.
(Bruno Roustant)
- SOLR-11706: Add support for aggregation on multivalued fields in JSON facets. min, max, avg, sum, sumsq, stddev,
variance, percentile aggregations now have support for multivalued fields.
(hossman, Munendra S N)
- SOLR-13987: Admin UI should not rely on javascript eval()
(rmuir, Kevin Risden)
- SOLR-13979: Expose separate metrics for distributed and non-distributed requests.
(ab)
- SOLR-13957: Add sensible defaults for the facet, random, facet2D, timeseries, stats
and update Streaming Expressions
(Joel Bernstein)
- SOLR-13904: Analytic component abandon requests exceedig limit passed via timeAllowed parameter
(Houston Putman, Mikhail Khludnev).
- SOLR-13970: Fail the request when collapsing or expand is used with Grouping.
(Erick Erickson, Joel Bernstein, Munendra S N)
- Bug Fixes (27)
- SOLR-13827: Fail on unknown operation in Request Parameters API
(Munendra S N, noble)
- SOLR-13403: Fix NPE in TermsComponent for DatePointField
(yonik, Munendra S N)
- SOLR-9802: Fix grouping failure for date field in solrcloud
(Erick Erickson, Munendra S N, Vitaly Lavrov)
- SOLR-12393: Compute score if requested even when expanded docs not sorted by score in ExpandComponent.
(David Smiley, Munendra S N)
- SOLR-13877, SOLR-7798: Robust support for ExpandComponent when used independently of CollapsingPostFilter.
(Jörg Rathlev, Michael Gibney, Munendra S N)
- SOLR-13823: Fix ClassCastEx when score is requested with group.query. This also fixes score not being generated
for distributed group.query case.
(Uwe Jäger, Munendra S N)
- SOLR-13207: In dismax and edismax "minimum should match" queries, the < operator not followed by a number will now return
an HTTP 400 error rather than 500, and the error message will explain that < must be followed by a number.
(Chris Hennick)
- SOLR-13762: Fix BinaryField support for non-binary wt's
(Thomas Woeckinger via Jason Gerlowski)
- SOLR-13882: Collections API COLSTATUS does not check live_nodes when reporting replica's status.
(Erick Erickson, Andrzej Białecki)
- SOLR-13872: Fixed Backup failures - typically manifesting as NoSuchFileException - due to race conditions
in saving/reserving commit points
(hossman)
- SOLR-13921: Processing UpdateRequest with delegation token throws NullPointerException
(Istvan Farkas via Kevin Risden)
- SOLR-13941: Tests configure Jetty differently than when running via start.jar
(janhoy, Uwe Schindler)
- SOLR-13950: Fix InterruptedException being swallowed in ZkStateReader’s getLeaderRetry method
(Andy Vuong via Tomás Fernández Löbbe)
- SOLR-13465: CoreContainer.auditloggerPlugin should be volatile
(janhoy, hossman)
- SOLR-13805: NPE when calling /solr/admin/info/health on standalone solr
(Nicholas DiPiazza, shalin)
- SOLR-13954: Embedded ZooKeeper in Solr now does not try to load JettyAdminServer
(janhoy)
- SOLR-13087: Use EUID instead of whoami to check for root in the bin/solr script
(Martijn Koster via Jason Gerlowski)
- SOLR-13977: solr create -c not working under Windows 10
(janhoy)
- SOLR-13966: LatLonPointSpatialField wasn't working correctly with atomic/partial updates or RTG.
(Thomas Wöckinger via David Smiley)
- SOLR-13563: SPLITSHARD using LINK method fails on disk usage checks.
(Andrew Kettmann, ab)
- SOLR-13806: SolrJ QueryResponse._explainMap is incorrectly typed.
(Guna Sekhar Dorai, ab)
- SOLR-13975, SOLR-13896: ConcurrentUpdateSolrClient connection stall prevention.
(ab, caomanhdat)
- SOLR-13953: Prometheus exporter in SolrCloud mode limited to 100 nodes
(Alex Jablonski via Erick Erickson)
- SOLR-13945: Fix: SPLITSHARD can cause data loss on a failure to commit after the sub-shards are active and a rollback
is done to make parent shard active again
(Ishan Chattopadhyaya, ab)
- SOLR-14071: Untrusted configsets cannot use <lib> directive due to security reasons
(Ishan Chattopadhyaya)
- SOLR-14013: FIX: javabin performance regressions
(noble, yonik, Houston Putman)
- SOLR-14079: SPLITSHARD splitByPrefix doesn't work in async mode. This also
affects splits triggered by the autoscale framework, which use async mode.
(Megan Carey, Andy Vuong, Bilal Waheed, Ilan Ginzburg, yonik)
- Other Changes (11)
- SOLR-12769: Fix incorrect documentation for 'delete' op in Request parameters API
(Alexandre Rafalovitch, Munendra S N)
- SOLR-13841: Provide mappings for jackson annotation @JsonProperty to use Jackson deserializer
(noble)
- SOLR-13824: Strictly reject anything after JSON in most APIs
(Mikhail Khludnev, Munendra S N)
- SOLR-13869: SolrTestCaseJ4.getNextAvailablePort() has been deprecated (and will be removed in 9.0) since
it is fundementally flawed. Users with tests that depend on this method should restructure them.
(hossman)
- SOLR-13782: Solr Ref Guide no longer published in PDF format.
(Cassandra Targett)
- SOLR-12193: Move some log messages to TRACE level
(gezapeti, janhoy)
- SOLR-13885: Typos in the documentation.
(KoenDG via Cassandra Targett)
- SOLR-13992: Code refactored to have collection name, slice name in Replica, Slice
(noble)
- SOLR-13998: Add thread safety annotations to Solr. This only introduces the annotations and doesn't add these to
existing classes.
(Anshum Gupta, Mark Miller)
- SOLR-13818: Upgrade jackson to 2.10.0
(marungo, Andras Salamon via Kevin Risden)
- SOLR-13999: Document zkHost param on topic() stream
(Vergil Cheynov via Jason Gerlowski)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.5.5
- Jetty 9.4.24.v20191120
- Upgrade Notes (1)
- Velocity: The initialization parameters
"params.resource.loader.enabled" and "solr.resource.loader.enabled" have been removed and replaced by
Java system properties "velocity.resourceloader.params.enabled" and "velocity.resourceloader.solr.enabled"
respectively (both turned off by default). Please see SOLR-13971 for more details.
- Bug Fixes (5)
- SOLR-13963: JavaBinCodec has concurrent modification of CharArr resulting in corrupt internode updates
(Colvin Cowie, noble)
- SOLR-13971: Velocity response writer's resource loading now possible only through startup parameters.
(Ishan Chattopadhyaya, David Smiley)
- SOLR-13465: CoreContainer.auditloggerPlugin should be volatile
(janhoy, hossman)
- SOLR-13905: Make findRequestType in AuditEvent more robust
(janhoy)
- SOLR-13941: Tests configure Jetty differently than when running via start.jar
(janhoy, Uwe Schindler)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.5.5
- Jetty 9.4.19.v20190610
- Upgrade Notes (2)
- Users who have written test cases that extend SolrTestCaseJ4 may see NullPointerExceptions if
their tests directly reference both SolrTestCaseJ4.initCoreDataDir and SolrTestCaseJ4.deleteCore().
This change in behavior is due to a bug fix in deleteCore() to ensure the dataDir is properly reset
in tests that call initCore()/deleteCore() multiple times in a given test (class). initCoreDataDir
is now deprecated, and users are encouraged to use SolrTestCaseJ4.initAndGetDataDir() in it's place.
See SOLR-13664 for more details.
- For JWTAuthPlugin, the 'jwkUrl' configuration key is deprecated and may be removed later, please use 'jwksUrl'
instead. See SOLR-13734.
- New Features (15)
- SOLR-13375: Two dimensional routed aliases are now available for organizing collections based on the data values
of two fields.
(Gus Heck)
- SOLR-13399: SPLITSHARD implements a new splitByPrefix option that takes into account the actual document distribution
when using compositeIds. Document distribution is calculated using the "id_prefix" field (if it exists) containing
just the compositeId prefixes, or directly from the indexed "id" field otherwise.
(yonik, Megan Carey)
- SOLR-13622: Add cat() stream source to create tuples from lines in local files
(Jason Gerlowski and Joel Bernstein)
- SOLR-11866: QueryElevationComponent can have query rules configured with match="subset" wherein the words need only
match a subset of the query's words and in any order.
(Bruno Roustant via David Smiley)
- SOLR-13682: command line option to export documents to a file
(noble)
- SOLR-13257: Support deterministic replica routing preferences for better cache usage
(Michael Gibney
via Christine Poerschke, Tomás Fernández Löbbe)
- SOLR-13122: Ability to query aliases in Solr Admin UI
(janhoy)
- SOLR-13713: JWTAuthPlugin to support multiple JWKS endpoints
(janhoy)
- SOLR-13734: JWTAuthPlugin now supports multiple IdP issuers through configuring a new 'issuers' configuration key.
Access tokens issued and signed by any of the configured issuers will be validated
(janhoy)
- SOLR-13272: Add support for arbitrary ranges in JSON facet's Range facets.
(Apoorv Bhawsar, Munendra S N, Mikhail Khludnev, Ishan Chattopadhyaya, Jan Høydahl)
- SOLR-13632: Support integral plots, cosine distance and string truncation with math expressions
(Joel Bernstein)
- SOLR-13667: Add upper, lower, trim and split Stream Evaluators
(Joel Bernstein)
- SOLR-13625: Add CsvStream, TsvStream Streaming Expressions and supporting Stream Evaluators
(Joel bernstein)
- SOLR-8241: Add CaffeineCache, an efficient implementation of SolrCache.(Ben Manes, Shawn Heisey, David Smiley, Andrzej Bialecki)
- SOLR-13298: Allow zplot to plot matrices
(Joel Bernstein)
- Improvements (19)
- SOLR-12368: Support InPlace DV updates for a field that does not yet exist in any documents
(hossman, Simon Willnauer, Adrien Grand, Munendra S N)
- SOLR-13558, SOLR-13693: Allow dynamic resizing of SolrCache-s.
(ab)
- SOLR-6305: Ability to set the replication factor for index files created by HDFSDirectoryFactory
(Boris Pasko via Kevin Risden)
- SOLR-13702: Some components register twice their metric names
(janhoy)
- SOLR-11601: Improved error message when geodist(llpsf) is used with arguments referring to a LatLonPointSpatialField.
(Amrit Sarkar)
- SOLR-13542: Code cleanup - Avoid using stream filter count where possible
(Koen De Groote via Tomás Fernández Löbbe)
- SOLR-13720: BlockJoinParentQParser.getCachedFilter()made public for accessing from QParser plugins
(Stanislav Livotov via Mikhail Khludnev)
- SOLR-13728: If a partial update (aka atomic update) is attempted on a document that has child docs, then ensure
the schema supports it (_root_ stored/docValues) by throwing an exception.
(David Smiley)
- SOLR-13742: Allow optional redaction of data saved by 'bin/solr autoscaling -save'.
(ab)
- SOLR-13739: Optimized large managed schema modifications; Internal O(n^2) problem.
(Thomas Wöckinger via David Smiley)
- SOLR-9658: Max idle time support for SolrCache implementations.
(hoss, ab)
- SOLR-13763: Improve the tracking of "freedisk" in autoscaling simulations.
(ab)
- SOLR-13773: Add Prometheus Exporter GC and Heap options.
(Houston Putman via Anshum Gupta, David Smiley)
- SOLR-13638: Add debug, trace logging to RuleBasedAuthorizationPlugin
(Jason Gerlowski)
- SOLR-13784: EmbeddedSolrServer's defaultCore constructor argument is now optional
(David Smiley)
- SOLR-13798: SSL: Adding Enabling/Disabling client's hostname verification config
(Cao Manh Dat)
- SOLR-13771: Add -v and -m to ulimit section of reference guide and bin/solr checks
(Erick Erickson)
- SOLR-13795: Managed schema operations should do a core reload in Solr standalone mode.
(Thomas Wöckinger via David Smiley)
- SOLR-13719: Introducing SolrClient.ping(collection) in SolrJ
(Geza Nagy via Mikhail Khludnev)
- Bug Fixes (42)
- SOLR-13206: Fix AIOOBE when group.facet is specified with group.query and return proper error code.
(Marek, Munendra S N)
- SOLR-11556: Backup and restore command really supports picking one of a few repositories by
repository parameter
(Timothy Potter via Mikhail Khludnev)
- SOLR-13647: Default solr.in.sh contains incorrect default value
(John Ryan)
- SOLR-13660: Fixed AbstractFullDistribZkTestBase.waitForActiveReplicaCount() to ensure
replicas are active.
(hossman)
- SOLR-13664: Fixed SolrTestCaseJ4.deleteCore() to properly reset the dataDir used by initCore()
(hossman)
- SOLR-13679: Default style of ExplainDocTransformer registered via solrconfig.xml should be same as default style
of ExplainDocTransformer registered in defaultFactories
(Munendra S N)
- SOLR-13672: Cloud -> Zk Status page now parses response from Zookeeper 3.5.5 correctly
(Jörn Franke, janhoy, Shawn Heisey)
- SOLR-13674: NodeAddedTrigger does not support configuration of replica type hint. A new replicaType property
has been added to NodeAddTrigger so that new replicas of the given type are added. The default value is `NRT`.
(Irena Shaigorodsky via shalin)
- SOLR-13141: CDCR bootstrap does not replicate index to the replicas of target cluster.
(Krzysztof Watral, Amrit Sarkar, Tim, Tdspringsteen, shalin)
- SOLR-13694: IndexSizeEstimator NullPointerException.
(ab, hossman)
- SOLR-13700: Fixed a race condition when initializing metrics for new security plugins on security.json change
(hossman)
- SOLR-13701: Fixed JWTAuthPlugin to update metrics prior to continuing w/other filters or returning error
(hossman)
- SOLR-6328: facet.missing=true should return missing count even when facet.limit=0
(hossman, Munendra S N)
- SOLR-13704: Improve error message and change error code to 400 for client errors in ExpandComponent
(Munendra S N)
- SOLR-13699 - maxChars no longer working on CopyField with javabin
(Chris Troullis via noble)
- SOLR-13718: SPLITSHARD (async) with failures in underlying sub-operations can result in data loss
(Ishan Chattopadhyaya)
- SOLR-13717: Fixed distributed grouping when multiple 'fl' params are specified
(hossman, Christine Poerschke)
- SOLR-13240: Fixed UTILIZENODE action resulting in IllegalArgumentException.
(Hendrik Haddorp, Richard Goodman, Tim Owen, shalin, noble, Christine Poerschke)
- SOLR-13238: BlobHandler generates non-padded md5
(Jeff Walraven via janhoy)
- SOLR-13780: Fix ClassCastException in NestableJsonFacet
(Tiago Martinho de Barros, Munendra S N)
- SOLR-13725: Allow negative values for limit in TermsFacetMap
(Richard Walker, Munendra S N)
- SOLR-13022: Fix NPE when sorting by non-existent aggregate function in JSON Facet
(hossman, Munendra S N)
- SOLR-13727: Fixed V2Requests - HttpSolrClient replaced first instance of "/solr" with "/api" which
caused a change in host names starting with "solr".
(Megan Carey via yonik)
- SOLR-13180: Fix ClassCastException in Json Request API
(Johannes Kloos, Jan Høydahl, Munendra S N)
- SOLR-13417: Handle stats aggregation on date and string fields in SolrJ's JSON facet response processing
(Jason Gerlowski, Munendra S N)
- SOLR-13712: JMX MBeans are not exposed because of race condition between creating platform mbean server and
registering mbeans.
(shalin)
- SOLR-13802: Managed schema manipulations were not persisting the optional luceneMatchVersion that can be set
on an Analyzer.
(Thomas Wöckinger)
- SOLR-13539: Fix for class-cast issues during atomic-update 'removeregex' operations. This also incorporated some
tests Tim wrote as a part of SOLR-9505.
(Tim Owen via Jason Gerlowski)
- SOLR-13790: LRUStatsCache size explosion and ineffective caching.
(ab)
- SOLR-13376: Multi-node race condition to create/remove nodeLost markers.
(hoss, ab)
- SOLR-13293: ConcurrentUpdateHttp2SolrClient always log AsynchronousCloseException exception error on indexing.
(Cao Manh Dat)
- SOLR-13828: Improve ExecutePlanAction error handling.
(ab)
- SOLR-13760: Fix regression in support for Date math in TRA start date that was introduced by SOLR-13375
- SOLR-13829: RecursiveEvaluator casts Continuous numbers to Discrete Numbers, causing mismatch
(Trey Grainger, Joel Bernstein)
- SOLR-13815: Live shard split (where updates actively continue during the split) can lose updates due to cluster
state happening to change between checking if the current shard is active and later checking if there are any
sub-shard leaders to forward the update to.
(yonik)
- SOLR-13665: Added missing netty dependencies to solrJ and upgraded netty to v 4.1.29.Final
(Jörn Franke, janhoy)
- SOLR-13793: HttpSolrCall now maintains internal request count (_forwardedCount) for remote queries and limits them to
the number of replicas. This avoids making too many cascading calls to remote servers, which, if not restricted, can
bring down nodes containing the said collection
(Kesharee Nandan Vishwakarma, Ishan Chattopadhyaya)
- SOLR-13834: ZkController#getSolrCloudManager() created a new instance of ZkStateReader, thereby causing mismatch in the
visibility of the cluster state and, as a result, undesired race conditions
(Clay Goddard via Ishan Chattopadhyaya)
- SOLR-13835: HttpSolrCall produces incorrect extra AuditEvent on AuthorizationResponse.PROMPT
(janhoy, hossman)
- SOLR-13843: The MOVEREPLICA API ignores replica type and always adds 'nrt' replicas
(Amrit Sarkar via shalin)
- SOLR-13677: All Metrics Gauges should be unregistered by components that registered them.
(noble, ab)
- SOLR-13855: DistributedZkUpdateProcessor should have been propagating URP.finish() lifecycle like it used to before
8.1 (a regression). Impacts integrity since Run URP's finish() propagates this to the updateLog to fsync.
(David Smiley)
- Other Changes (16)
- SOLR-13779: Use the safe fork of simple-xml for clustering contrib.
(Dawid Weiss)
- SOLR-13585: Factor out SearchGroupsResultTransformer.[de]serializeOneSearchGroup methods.
(Christine Poerschke, Diego Ceccarelli)
- SOLR-12870: Use StandardCharsets instead of String values
(Peter Somogyi via Munendra S N)
- SOLR-10377: Add `debug.explain.structured` to Admin UI.
(David Smiley, Munendra S N)
- SOLR-13629: Clean trailing whitespace from 'analytics' contrib
(Neal Sidhwaney via Jason Gerlowski)
- SOLR-13643: Add Getters/Setters in ResponseBuilder for analytics response handling
(Neal Sidhwaney via Munendra S N)
- SOLR-13680: Use try-with-resource to close the closeable resource
(Furkan KAMACI, Munendra S N)
- SOLR-13573: Add SolrRangeQuery getters for upper, lower bound
(Brian Rhees via Jason Gerlowski)
- SOLR-13658: Precommit fail Java "var" until 9x. Fail "var...<>" constructs entirely
(Erick Erickson)
- SOLR-13767: Upgrade jackson to 2.9.9
(janhoy)
- SOLR-11492: Clean up /solr/cloud-dev scripts and provide a single well documented script
(Gus Heck, Robert Bunch)
- SOLR-13747: New TestSSLTestConfig.testFailIfUserRunsTestsWithJVMThatHasKnownSSLBugs() to give people running
tests more visibility if/when they use a known-buggy JVM causing most SSL tests to silently SKIP.
(hossman)
- SOLR-13791: Remove remaining Commons BeanUtils references.
(Andras Salamon, Christine Poerschke)
- SOLR-13812: Add javadocs, uneven rejection and basic test coverage for the SolrTestCaseJ4.params method.
(Diego Ceccarelli, Christine Poerschke, Munendra S N)
- SOLR-13787: An annotation based system to write v2 APIs
(noble)
- SOLR-12786: Update Ref Guide build tool versions
(Cassandra)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.5.5
- Jetty 9.4.19.v20190610
- Upgrade Notes (6)
- SOLR-8346: Upgrade ZooKeeper to 3.5.5. ZooKeeper 3.5.5 introduces more security features. Include in your zoo.cfg
file at minimum the following:
4lw.commands.whitelist=mntr,conf,ruok
You can use
4lw.commands.whitelist=* to enable all ZooKeeper "4 letter commands".
(Erick Erickson)
- SOLR-13515: org.apache.solr.util.SolrPluginUtils.IdentityRegenerator has been removed, please use
the identical org.apache.solr.search.NoOpRegenerator instead.
(David Smiley, Christine Poerschke)
- SOLR-13541: Upgrade Jetty to 9.4.19.v20190610.
(Erick Erickson, Cao Manh Dat)
- SOLR-13420: Routed Aliases now use collection properties rather than core properties to identify collections that
belong to the alias by default. This should be invisible and fully backwards compatible from within solr, and
existing routed alias collections with core based properties will continue to work, but new collections created will
not add a property to core.properties anymoore so any external code that inspected core.properties will not find the
'routedAliasName' key in new cores belonging to routed aliases.
- SOLR-13419: Time Routed Aliases now have a '__TRA__' infix preceding the timestamp portion of the collection names.
collections with the old format will not be altered and will continue to work, but any new TRA's or new collections
for existing TRA's will use the new format. Solr will handle this invisibly, but any external code that attempted to
predict collection names in TRA's will probably need adjustment.
- SOLR-13507: Support for "addr" parameter from the "/solr/admin/zookeeper" endpoint has now been removed and will no
longer be supported.
- New Features (11)
- SOLR-13320: add an update param failOnVersionConflicts=false to updates not fail when there is a version conflict
(noble)
- SOLR-13445: Preferred replicas on nodes with same system properties as the query master
(Cao Manh Dat)
- SOLR-13047: Add facet2D Streaming Expression
(Nazerke Seidan, Joel Bernstein)
- SOLR-13494: Add DeepRandomStream implementation
(Joel Bernstein)
- SOLR-13434: OpenTracing support for Solr
(Cao Manh Dat)
- SOLR-13512: Raw index data analysis tool (extension of COLSTATUS collection command).
(ab)
- SOLR-13550: Allow zplot to automatically create the x axis
(Joel Bernstein)
- SOLR-13552: Add recNum Stream Evaluator
(Joel Bernstein)
- SOLR-13560: Add isNull and notNull Stream Evaluators
(Joel Bernstein)
- SOLR-10291: Add matches Stream Evaluator to support regex matching
(Joel Bernstein)
- SOLR-13589: Allow zplot to visualize 2D clusters and convex hulls
(Joel Bernstein)
- Improvements (16)
- SOLR-13049: Make contrib/ltr Feature.defaultValue configurable.
(Stanislav Livotov, Christine Poerschke)
- SOLR-13306: Add a request parameter to execute a streaming expression locally
- SOLR-13440: Support saving/restoring autoscaling state for repeatable simulations.
(ab)
- SOLR-11558: It would be nice if the Graph section of the Cloud tab in the Admin UI could give some more
information about the replicas of a collection
(Erick Erickson)
- SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload
(noble)
- SOLR-12304: The MoreLikeThisComponent now supports the mlt.interestingTerms parameter. Previously this option was
unique to the MLT handler.
(Alessandro Benedetti via David Smiley)
- SOLR-13484: autoscaling/diagnostics API should be able to give diagnostics output from config sent as a payload
(noble)
- SOLR-13493: /autoscaling/suggestions to be able to filter by type
(noble)
- SOLR-13504: In autoscaling policies, use an explicit 'nodeset' attribute for filtering
nodes instead of using them directly at the toplevel
(noble)
- SOLR-13329: In autoscaling policies, use an explicit 'put : on-each-node'
to specify the the rules is applied on each node
(noble)
- SOLR-13347: Transaction log to natively support UUID types
(Thomas Wöckinger via noble)
- SOLR-13367: Highlighting: Range queries will now highlight in hl.method=unified mode.
(David Smiley)
- SOLR-13569: Admin UI visual indication of prod/test/dev environment
(janhoy)
- SOLR-12554: Expose IndexWriterConfig's ramPerThreadHardLimitMB through solrconfig.xml
(Ishan Chattopadhyaya, Munendra S N)
- SOLR-13602 SOLR-13588: Add a field type for Estonian language to default managed_schema,
document about Estonian language analysis in Solr Ref Guide
(Tomoko Uchida)
- SOLR-13003: Query Result Cache does not honor maxRamBytes parameter.
(ab, Brian Ecker)
- Bug Fixes (24)
- SOLR-13474: Fix "Search is temporarily disabled" logic to be consistent for entire request
(hossman)
- SOLR-13469: Fix rejected requests during full disk full + IndexFetch to use 503 not 403
(hossman)
- SOLR-12941: Fix IndexSizeTrigger to correctly work with "aboveBytes" and "splitMethod=link" parameters.
(ab)
- SOLR-13491: SolrZkClient's watch wrapper no longer allows zookeeper to hold the same watch object multiple times.
- SOLR-13489: Stop the leader from trying to rejoin the election on session expiration and harden our zk reconnect code path.
(Mark Miller, Anshum Gupta)
- SOLR-12249: Better error message when grouping on a tokenized (non SortableText) field in SolrCloud
(Erick Erickson)
- SOLR-13521: Fix input parameter handling in SchemaRequest.DynamicField and SchemaRequest.FieldTypes (Schema API)
(Tomás Fernández Löbbe)
- SOLR-13496: Fix distributed grouping related NullPointerException in JSONWriter.writeSolrDocument
(Christine Poerschke)
- SOLR-12013: collections API CUSTERSTATUS command fails when configset missing
(Erick Erickson)
- SOLR-13509: NPE on omitHeader=true is fixed by sending omitHeader=false to shard searches
(Munendra S N, Mikhail Khludnev)
- SOLR-13333: unleashing terms.ttf from terms.list when distrib=false
(Munendra S N via Mikhail Khludnev)
- SOLR-13490: Fix CollectionStateWatcher/CollectionStatePredicate based APIs in ZkStateReader and
CloudSolrClient to be triggered on liveNode changes. Also add Predicate<DocCollection> equivalents
for callers that don't care about liveNodes.
(hossman)
- SOLR-13545: ContentStreamUpdateRequest refused to close file
(Colvin Cowie, Mikhail Khludnev)
- SOLR-12127: Updates containing 'set' operation with value null or empty list should be considered as Atomic Update
(Oliver Kuldmäe, Munendra S N, Ishan Chattopadhyaya)
- SOLR-13187: Fix NPE when invalid query parser is specified and return 400 error code.
(Cesar Rodriguez, Marek, Charles Sanders, Munendra S N, Mikhail Khludnev)
- SOLR-12979: Improve error message and change error code to 400 when collapse field is non-docValued and
non-uninvertible
(hossman, Munendra S N)
- SOLR-13574: Fix many test and test-framework classes to not fail on After/AfterClass cleanup if
assumptions fail in Before/BeforeClass setup
(hossman)
- SOLR-13280: Strengthen ScheduledTrigger's preferredOperation parameter validation.
(Christine Poerschke)
- SOLR-9409: Improve error message for unsupported field types and return 400 error code on
unsupported values in collapsing
(hossman, Munendra S N)
- SOLR-13404: Support group.query in multi-shard environment when group.main=true or group.format=simple
(Munendra S N)
- SOLR-13538: toNativeType () TrieDate & EnumField do not handle CharSequence properly
(Munendra S N)
- SOLR-13532: Fix http timeout and error logging bugs in RecoveryStrategy.
(Suril Shah, hossman)
- SOLR-13472: Forwarded requests should skip authorization on receiving nodes
(adfel, Ishan Chattopadhyaya)
- SOLR-13619: Kerberos plugin to pass original user principal to avoid 403 on nodes not hosting a collection
(adfel, Ishan Chattopadhyaya, noble)
- Other Changes (15)
- SOLR-13453: Adjust auth metrics asserts in tests caused by SOLR-13449
(janhoy)
- SOLR-13437: noggit json parser is forked into solrj
(noble)
- SOLR-13467: Include the S2 Geometry lib to make it simpler to use prefixTree="s2" on a Geo3D spatial field.
Improved documentation on Geo3D too.
(David Smiley)
- SOLR-13371: Re-structure and clarify Security chapters in Ref Guide
(janhoy)
- SOLR-13235: Split Collections API Ref Guide page into several smaller child pages
(Cassandra Targett)
- SOLR-8754: Adding test cases for org.apache.solr.util.hll.NumberUtilTest
(Benoit Vanalderweireldt via janhoy)
- SOLR-13511: Add SearchHandler.newResponseBuilder method to facilitate custom plugins' maintenance
of per-request state in a custom ResponseBuilder.
(Ramsey Haddad, Christine Poerschke)
- SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds
request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs
(hossman, Cao Manh Dat)
- SOLR-13279: Clarify ScheduledTrigger's "every parameter missing" error response.
(Christine Poerschke)
- SOLR-13576: Factor out a TopGroupsShardResponseProcessor.fillResultIds method.
(Christine Poerschke, Diego Ceccarelli)
- SOLR-12364: Add test cases for edismax boost
(David Smiley, Munendra S N)
- SOLR-13603: Remove usage of GroupingSpecification's deprecated methods
(Munendra S N)
- SOLR-13507: Remove support for "addr" parameter from the "/solr/admin/zookeeper" endpoint.
(janhoy, Anshum Gupta)
- SOLR-13588: Document Estonian analyzer in Solr Ref Guide
(Tomoko Uchida)
- SOLR-13602: Add a field type for Estonian language to default managed_schema
(Tomoko Uchida)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.4.14
- Jetty 9.4.14.v20181114
- Bug Fixes (5)
- SOLR-13510: Intermittent 401's for internode requests with basicauth enabled
(Cao Manh Dat, Colvin Cowie)
- SOLR-13523: In 8.1, Atomic Updates were broken (NPE) when the schema declared the new _nest_path_ field even if you
weren't using nested docs. In-place updates were not affected (worked).
Reminder: if you don't need nested docs then remove both _root_ and _nest_path_ !
(David Smiley)
- SOLR-13566: REINDEXCOLLECTION does not work with (basic) authentication.
(Colvin Cowie, ab)
- SOLR-13159: Fix atomic update encoding issue for UUID, enum, bool, and binary fields
(Thomas Wockinger via Jason Gerlowski)
- SOLR-13583: Impossible to delete a collection with the same name as an existing alias. This fixes also a bug in
REINDEXCOLLECTION when used with removeSource=true which could lead to a data loss.
(ab)
- Improvements (1)
- SOLR-13158: DataImportHandler: Added enable.dih.dataConfigParam system property to toggle whether the dataConfig param
is permitted.
(David Smiley, janhoy, Tomás Fernández Löbbe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.4.14
- Jetty 9.4.14.v20181114
- Bug Fixes (1)
- SOLR-13475: Null Pointer Exception when querying collection through collection alias.
(Jörn Franke, ab)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 2.0 and Velocity Tools 3.0
- Apache ZooKeeper 3.4.14
- Jetty 9.4.14.v20181114
- Upgrade Notes (8)
- Solr's default behavior when dealing with 'maxBooleanClauses' has changed to reduce the risk of exponential
query expansion when dealing with pathological query strings. A default upper limit of 1024 clauses
(The same default prior to Solr 7.0) is now enforced at the node level, and can be overridden in solr.xml.
The identically named solrconfig.xml setting is still available for limiting the size of 'explicit' boolean
query strings, but this per-collection limit is still ristricted by the upper-bound of the global limit
in solr.xml. See SOLR-13336 for more details.
- When requesting the status of an async request via REQUESTSTATUS collections API, the response will
include the list of internal async requests (if any) in the "success" or "failed" keys (in addition
to them being included outside those keys for backwards compatibility). See SOLR-12708 for more
details
- SOLR-13335: Velocity and Velocity Tools were both upgraded as part of this release. Velocity upgraded from 1.7 to 2.0.
Please see https://velocity.apache.org/engine/2.0/upgrading.html about upgrading. Velocity Tools upgraded from 2.0 to
3.0. For more details, please see https://velocity.apache.org/tools/3.0/upgrading.html for details about the upgrade.
- SOLR-13407: Update requests sent to non-routed aliases that point to multiple collections are no longer accepted. Until
now Solr followed an obscure convention of updating only the first collection from the list, which usually was not what
the user intended. This change explicitly rejects such update requests.
- SolrGangliaReporter has been removed from Solr because support for Ganglia has been removed from Dropwizard Metrics 4
due to a transitive dependency on LGPL.
- Custom TransientSolrCoreCache implementations no longer use the Observer/Observable pattern. To notify Solr that
a core has been aged out of the cache, call CoreContainer.queueCoreToClose(SolrCore). See SOLR-13400 for details.
- SOLR-13394: The default GC has been changed from CMS to G1. To override this (in order to switch to CMS or any
other GC), use GC_TUNE section of bin/solr.in.sh or bin/solr.in.cmd.
- SOLR-5970: Until now, the CREATE command of Collections API returned status 0 even in case of failure. Now, the status
code will be non-zero in case of failures (e.g. 4xx, 5xx).
- New Features (16)
- SOLR-13131: Category Routed Aliases are now available for data driven assignment of documents to collections based on
values of a field. The Ref Guide now has a page dedicated to explaining the different types of aliases.
(Gus Heck,
Moshe Bla)
- SOLR-13171 : A true streaming parser for javabin payload/stream without creating any objects
(noble)
- SOLR-13261: Make SortableTextField work with export/streaming. NOTE: requires that the field have
useDocValuesAsStored=true (either explicit or as the default).
- SOLR-12121: JWT Token authentication plugin with OpenID Connect implicit flow login through Admin UI
(janhoy)
- SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins
(janhoy)
- SOLR-10436: Add hashRollup Streaming Expression
(Joel Bernstein)
- SOLR-13276: Adding Http2 equivalent classes of CloudSolrClient and HttpClusterStateProvider
(Cao Manh Dat)
- SOLR-13287: Allow zplot to visualize probability distributions in Apache Zeppelin
(Joel Bernstein)
- SOLR-13271: Read-only mode for SolrCloud collections
(ab, shalin)
- SOLR-13292: Provide extended per-segment status of a collection.
(ab)
- SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. This issue also adds
a back-compat check of the .system collection to notify users of potential compatibility issues after
upgrades or schema changes.
(ab)
- SOLR-13374: Add fetchSize parameter to the jdbc Streaming Expression
(Joel Bernstein)
- SOLR-12638: Partial/Atomic Updates for nested documents. This enables atomic updates for nested documents, without
the need to supply the whole nested hierarchy (which would be overwritten if absent). This is done by fetching the
whole document hierarchy, updating the specific doc in the path that is to be updated, removing the old document
hierarchy and indexing the new one with the atomic update merged into it. Also, [child] Doc Transformer now works
with RealTimeGet.
(Moshe Bla, David Smiley)
- SOLR-13262: Add collection RENAME command and support using aliases in most collection admin commands.
(ab)
- SOLR-13391: Add variance and standard deviation stream evaluators
(Nazerke Seidan, Joel Bernstein)
- SOLR-13427: Support simulating the execution of autoscaling suggestions.
(ab)
- Bug Fixes (39)
- SOLR-12330: 500 error code on json.facet syntax errors
(Munendra S N, Mikhail Khludnev)
- SOLR-13229: Cleanup replicasMetTragicEvent after all types of exception
(Tomás Fernández Löbbe)
- SOLR-11876: In-place update fails when resolving from Tlog if schema has a required field
(Justin Deoliveira, janhoy,
Ishan Chattopadhyaya)
- SOLR-12708: Async collection actions should not hide internal failures
(Mano Kovacs, Varun Thacker, Tomás Fernández Löbbe)
- SOLR-11883: 500 code on functional query syntax errors and parameter dereferencing errors
- (Munendra S N via Mikhail Khludnev)
- SOLR-13234: Prometheus Metric Exporter not threadsafe. This changes the prometheus exporter to collect metrics
from Solr on a fixed interval controlled by this tool and prevents concurrent collections. This change also improves
performance slightly by using the cluster state instead of sending multiple HTTP requests to each node to lookup
all the cores.
(Danyal Prout via shalin)
- SOLR-9882: 500 error code on breaching timeAllowed by core and distributed (fsv) search,
old and json facets
(Mikhail Khludnev)
- SOLR-13285: Updates with enum fields and javabin cause ClassCastException
(noble)
- SOLR-13295: Reproducible failure in TestDistributedGrouping
(Erick Erickson)
- SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an exception
occurred while reconnecting to ZooKeeper.
(hu xiaodong via Christine Poerschke)
- SOLR-13284: NullPointerException with 500 http status on omitted or wrong wt param.
It's fixed by fallback to json
(Munendra S N via Mikhail Khludnev)
- SOLR-13244: Admin UI Nodes view fails and is empty when a node is temporarily down
(janhoy)
- SOLR-13253: IndexSchema.getResourceLoader was being used to load non-schema things, which can be a memory leak if
"shareSchema" and other circumstances occur. Furthermore it's reference to SolrConfig was removed.
(David Smiley)
- SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with '*' selector
(Michael Lawrence, Munendra S N, Ishan Chattopadhyaya)
- SOLR-13351: Workaround for VELOCITY-908
(Kevin Risden)
- SOLR-13349: High CPU usage in Solr due to Java 8 bug
(Erick Erickson)
- SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin
(janhoy, Jason Gerlowski)
- SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread).
(hossman)
- SOLR-13362: Add 'includeIndexFieldFlags' support to SolrJ LukeRequest
(Jason Gerlowski)
- SOLR-13355: 'all' permission ignored by RuleBasedAuthorizationPlugin in most cases
(Jason Gerlowski, janhoy)
- SOLR-13331: Atomic Update 'remove' operations broken for certain field types in SolrJ
(Thomas Wockinger via Jason Gerlowski)
- SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in schema.xml
(Uwe Schindler)
- SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread
(janhoy, Lorenzo)
- SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores already closed
(Cao Manh Dat)
- SOLR-13393: Fixed ZkClientClusterStateProvider to prevent risk of leaking ZkStateReader/threads when
processing concurrent requests during shutdown. This primarily affected tests, but may have also caused
odd errors/delays when restart/shutting down solr nodes.
(hossman)
- SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml, reverting previous effective
value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion.
(hossman)
- SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists
check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException.
(Mark Miller, Fernandez-Lobbe, Mike Drob)
- SOLR-12371: Editing authorization config via REST API now works in standalone mode
(janhoy)
- SOLR-13408: Cannot start/stop DaemonStream repeatedly, other API improvements
(Erick Erickson)
- SOLR-13281: Fixed NPE in DocExpirationUpdateProcessorFactory
(Munendra S N, Tomás Fernández Löbbe)
- SOLR-13081: In-Place Update doesn't work with route.field
(Dr Oleg Savrasov via Mikhail Khludnev)
- SOLR-13343: Fix spacing issue in browser log
(Marcus Eagan via Jason Gerlowski)
- SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields (non-stored, non-indexed)
(Erick Ericsson, Adrien Grand, Munendra S N, Scott Stults, Ishan Chattopadhyaya)
- SOLR-5970: Return correct status upon collection creation failure
(Abraham Elmahrek, Ishan Chattopadhyaya,
Jason Gerlowski, Kesharee Nandan Vishwakarma)
- SOLR-12291: prematurely reporting not yet finished async Collections API call as completed
when collection's replicas are colocated at least at one node
(Varun Thacker, Mikhail Khludnev)
- SOLR-13410: Designated overseer wasn't able to rejoin election queue upon restart
(Ishan Chattopadhyaya,
Kesharee Nandan Vishwakarma)
- SOLR-13318: Fix ClassCastException in SolrJ JsonFaceting classes
(Munendra S N via Jason Gerlowski)
- SOLR-13449: SolrClientNodeStateProvider always retries on requesting metrics from other nodes
(Cao Manh Dat)
- Improvements (14)
- SOLR-12999: Index replication could delete segments before downloading segments from master if there is not enough
disk space
(noble)
- SOLR-13227: Optimizing facet.range.other by avoiding expensive exceptions
(Nikolay Khitrin via Mikhail Khludnev)
- SOLR-9079: Remove commons-lang as a dependency
(Kevin Risden)
- SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
(Kevin Risden)
- SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/)
(Kevin Risden)
- SOLR-13398: Move log line "Processing SSL Credential Provider chain..." from INFO to DEBUG
(janhoy)
- SOLR-13407: Reject update requests sent to non-routed multi collection aliases.
(ab)
- SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload,
added test band-aid for DocValuesNotIndexedTest.
- SOLR-13337: Only request the minimum required number of terms from each shard when using terms.sort=index and none
are discarded due to terms.min/maxcount
(Morten Bøgeskov,Munendra S N via Mikhail Khludnev)
- SOLR-12167: Throw an exception, instead of just a warning, when unknown atomic update operation is
encountered
(Munendra S N via Ishan Chattopadhyaya)
- SOLR-13394: Switch default GC from CMS to G1
(Ishan Chattopadhyaya, Kesharee Nandan Vishwakarma, Shawn Heisey,
Uwe Schindler, Erick Ericsson, shalin)
- SOLR-13432: Add .toString methods to BitDocSet and SortedIntDocSet so that enabling "showItems" on the filter caches
shows some useful information about the values in the cache.
(shalin)
- SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used.
(jefferyyuan, ab)
- SOLR-13348: Speed up collapsing by avoiding scoring of ineligible documents
(Andrzej Wislowski via
Ishan Chattopadhyaya)
- Other Changes (27)
- SOLR-11763: Upgrade Guava to 25.1-jre
(Markus Jelsma, Kevin Risden)
- SOLR-13222: Improve logging in StreamingSolrClients
(Peter Cseh via Kevin Risden)
- SOLR-12055: Enable async logging by default. This change improves throughput for logging. This opens
up a small window where log messages could possibly be lost. If this is unacceptable, switching back to synchronous
logging can be done by changing the log4j2.xml file, no internal Solr code changed to make async logging the default.
(Erick Erickson)
- SOLR-12753: Async logging ring buffer and OOM error. When very long messages are written (1M messages or so),
it can produce an OOM error. Log messages are truncated at 10K via configuration in the log4j2.xml files.
- SOLR-9763: Remove the workaround implemented for HADOOP-12767
(Kevin Risden)
- SOLR-13060: Improve HdfsAutoAddReplicasIntegrationTest and HdfsCollectionsAPIDistributedZkTest
(Kevin Risden)
- SOLR-13074: MoveReplicaHDFSTest leaks threads, falls into an endless loop, logging like crazy
(Kevin Risden)
- SOLR-9762: Remove the workaround implemented for HADOOP-13346
(Kevin Risden)
- SOLR-7321: Remove reflection in FSHDFSUtils.java
(Mike Drob, Kevin Risden)
- SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(Erick Erickson)
- SOLR-13307: Ensure HDFS tests clear System properties they set
(Kevin Risden)
- SOLR-13330: Improve HDFS tests
(Kevin Risden)
- SOLR-8033: Remove debug if branch in HdfsTransactionLog
(Kevin Risden)
- SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud functionality into a subclass.
(Bar Rotstein, David Smiley)
- SOLR-13342: Remove dom4j from Solr
(Kevin Risden)
- SOLR-13335: Upgrade to velocity 2.0 and velocity-tools 3.0
(Kevin Risden)
- SOLR-13112: Upgrade jackson to 2.9.8
(Kevin Risden)
- SOLR-13353: Add SolrCli AuthTool test
(Kevin Risden)
- SOLR-13363: Upgrade to ZooKeeper 3.4.14
(Erick Erickson)
- SOLR-12809: Document recommended Java/Solr combinations
(Erick Erickson, Jan Høydahl et.al.)
- SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig
(Christine Poerschke)
- SOLR-13409: Disable HTML directory listings in admin interface to prevent possible security issues
(Uwe Schindler)
- SOLR-12461: Upgrade Dropwizard Metrics to 4.0.5 release.
(ab)
- SOLR-13400: Replace Observable pattern in TransientSolrCoreCache
(Erick Erickson)
- SOLR-13425: Ref-Guide: Wrong color of text in left column of horizontal definition list
(janhoy)
- SOLR-13423: Upgrade RRD4j to version 3.5.
(ab)
- SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
(Kevin Risden)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.13
- Jetty 9.4.14.v20181114
- Upgrade Notes (20)
- Due to the introduction of LIR redesign since Solr 7.3 (SOLR-11702) and the removing of old LIR implementation in Solr 8.
Rolling updates are not possible unless all nodes must be on Solr 7.3 or higher. If not updates can be lost.
- Solr nodes can now listen and serve HTTP/2 requests. Most of internal requests (sent by UpdateShardHandler, HttpShardHandler)
Http2SolrClient is used. Since by default, internal requests are sent by using HTTP/2,
Solr 8.0 nodes can't talk to old nodes (7.x). However we can follow these steps to do rolling updates:
-
Do rolling updates as normally, but the Solr 8.0 nodes must start with -Dsolr.http1=true as startup parameter.
By using this parameter internal requests are sent by using HTTP/1.1
-
When all nodes are upgraded to 8.0, restart them, this time -Dsolr.http1 parameter should be removed.
- ALPN is not supported in Java 8 or lower version therefore when a node started in Java 8 or a lower version with SSL enabled,
it will send and can only be able to handle HTTP/1.1 requests. In case of using SSL Java 9 or latter versions are recommended.
- Custom AuthenticationPlugin must provide its own setup for Http2SolrClient through
implementing HttpClientBuilderPlugin.setup, if not internal requests can't be authenticated.
- LUCENE-7996: The 'func' query parser now returns scores that are equal to 0
when a negative value is produced. This change is due to the fact that
Lucene now requires scores to be positive.
(Adrien Grand)
- SOLR-11882: SolrMetric registries retained references to SolrCores when closed. A
change of SolrMetricMAnager.registerGauge and SolrMetricProducer.initializeMetrics
method signatures was required to fix it. Third party components that use this API
need to be updated.
(Eros Taborelli, Erick Erickson, ab)
- LUCENE-8267: Memory codecs have been removed from the codebase (MemoryPostings,
MemoryDocValues). If you used postingsFormat="Memory" or docValuesFormat="Memory"
then either remove it to use the default or experiment with one of the others.
(Dawid Weiss)
- SOLR-12586: The date format patterns used by ParseDateFieldUpdateProcessorFactory (present in "schemaless mode")
are now interpreted by Java 8's java.time.DateTimeFormatter instead of Joda Time. The pattern language is very
similar but not the same. Typically, simply update the pattern by changing an uppercase 'Z' to lowercase 'z' and
that's it. For the current recommended set of patterns in schemaless mode, see "Schemaless Mode" in the ref guide,
or simply examine the default configSet. Also note that the set of patterns (formats) here have
expanded from before to subsume those patterns previously handled by the "extract" contrib (Solr Cell / Tika).
(David Smiley, Bar Rotstein)
- SOLR-12593: The "extraction" contrib (Solr Cell) no longer does any date parsing, and thus no longer has the
"date.formats" configuration. To ensure date strings are properly parsed, use ParseDateFieldUpdateProcessorFactory
(an URP) commonly registered with the name "parse-date" in "schemaless mode".
(David Smiley, Bar Rotstein)
- SOLR-12643: Since Http2SolrClient does not support exposing connections related metrics. These metrics are no longer
available 'QUERY.httpShardHandler.{availableConnections, leasedConnections, maxConnections, pendingConnections}',
'UPDATE.updateShardHandler.{availableConnections, leasedConnections, maxConnections, pendingConnections}'
- SOLR-12605: UpdateShardHandler's updateOnlyClient is now a Http2SolrClient (previous HttpSolrClient). This new
client does not support 'maxConnections','maxConnectionsPerHost' parameters.
- SOLR-12640: HttpShardHandlerFactory's defaultClient is now a Http2SolrClient (previous HttpSolrClient). This new
client does not support 'maxConnections','maxConnectionsPerHost' parameters. LBHttpSolrClient.Req and LBHttpSolrClient.Rsp
are marked as deprecated, uses LBSolrClient.Req and LBSolrClient.Rsp instead.
- SOLR-12754: The UnifiedHighlighter hl.weightMatches now defaults to true. If there are unforseen highlight problems,
this may be the culprit.
- If you explicitly use BM25SimilarityFactory in your schema, the absolute scoring will be lower due to SOLR-13025.
But ordering of documents will not change in the normal case. Use LegacyBM25SimilarityFactory if you need to force
the old 6.x/7.x scoring. Note that if you have not specified any similarity in schema or use the default
SchemaSimilarityFactory, then LegacyBM25Similarity is automatically selected for 'luceneMatchVersion' < 8.0.0.
See also explanation in Reference Guide chapter "Other Schema Elements".
- SOLR-12535: Solr no longer accepts index time boosts in JSON provided to Solr. This used to be provided like so:
{'id':'1', 'val_s':{'value':'foo', 'boost':2.0}} but will now produce an error. A object/map structure will now only
be interpreted as a child document or an atomic update; nothing else. A uniqueKey is currently required on all child
documents to be interpreted as such, though this may change in the future.
(David Smiley)
- SOLR-12633: When JSON data is sent to Solr with nested child documents split using the "split" parameter, the child
docs will now be associated to their parents by the field/label string used in the JSON instead of anonymously. Most
users probably won't notice the distinction since the label is lost any way unless special fields are in the schema.
This choice used to be toggleable with an internal/expert "anonChildDocs" parameter flag which is now gone.
(David Smiley)
- SOLR-11774: In 'langid' contrib, the LanguageIdentifierUpdateProcessor base class changed some method signatures.
If you have a custom language identifier implementation you will need to adapt your code.
- SOLR-9515: Hadoop dependencies have been upgraded to Hadoop 3.2.0 from 2.7.2.
(Mark Miller, Kevin Risden)
- SOLR-5211: Deleting (or updating) documents by their uniqueKey is now scoped to only consider root documents, not
child/nested documents. Thus a delete-by-id won't work on a child doc (no-op), and an attempt to update a child doc
by providing a new doc with the same ID would add a new doc (probably erroneous). Both these actions were and still
are problematic. In-place-updates are safe though. If you want to delete certain child documents and if you know
they don't themselves have nested children then you must do so with a delete-by-query technique.
- SOLR-13248: The default replica placement strategy used in Solr has been reverted to the 'legacy' policy used by Solr
7.4 and previous versions. This is due to multiple bugs in the autoscaling based replica placement strategy that was
made default in Solr 7.5 which causes multiple replicas of the same shard to be placed on the same node in addition
to the maxShardsPerNode and createNodeSet parameters being ignored. Although the default has changed, autoscaling
will continue to be used if a cluster policy or preference is specified or a collection level policy is in use.
The default replica placement strategy can be changed to use autoscaling again by setting a cluster property:
curl -X POST -H 'Content-type:application/json' --data-binary '
{
"set-obj-property": {
"defaults" : {
"cluster": {
"useLegacyReplicaAssignment":false
}
}
}
}' http://$SOLR_HOST:$SOLR_PORT/api/cluster
- New Features (14)
- SOLR-12591: Expand the set of recognized date format patterns of schemaless mode to subsume those handled by the
"extract" contrib (Solr Cell / Tika). This is primarily a change in configuration of the default configSet for more
patterns, but also included enabling "lenient" parsing in ParseDateFieldUpdateProcessorFactory. The default
locale was changed from ROOT to en_US since well-known patterns assume this locale.
(David Smiley, Bar Rotstein)
- SOLR-12879: MinHash query parser that builds queries providing a measure of Jaccard similarity
(Andy Hind via Tommaso Teofili)
- SOLR-12593: The default configSet now includes an "ignored_*" dynamic field.
(David Smiley)
- SOLR-12799: Allow Authentication Plugins to intercept internode requests on a per-request basis.
The BasicAuth plugin now supports a new parameter 'forwardCredentials', and when set to 'true',
user's BasicAuth credentials will be used instead of PKI for client initiated internode requests.
(janhoy, noble)
- SOLR-12791: Add Metrics reporting for AuthenticationPlugin
(janhoy)
- SOLR-12730: Implement staggered SPLITSHARD requests in IndexSizeTrigger.
(ab)
- SOLR-12639: Umbrella JIRA for adding support HTTP/2
(Cao Manh Dat)
- SOLR-12768, SOLR-13129: Improved nested document support, and enabled in the default schema with the presence of _nest_path_.
When this field is present, certain things happen automatically. An internal URP is automatically used to populate
it. The [child] (doc transformer) will return a hierarchy with relationships; no params needed. The relationship
path is indexed for use in queries (can be disabled if not needed). Also, child documents needn't provide a uniqueKey
value as Solr will supply one automatically by concatenating a path to that of the parent document's key.
(David Smiley, Moshe Bla).
- SOLR-13134: Allow the knnRegress Stream Evaluator to more easily perform bivariate regression.
(Joel Bernstein)
- SOLR-13104: Add natural and repeat Stream Evaluators
(Joel Bernstein)
- SOLR-13147: Add movingMAD Stream Evaluator
(Joel Bernstein)
- SOLR-13155: Add command-line option for testing autoscaling configurations.
(ab)
- SOLR-13241: Add 'autoscaling' tool support to solr.cmd
(Jason Gerlowski)
- SOLR-11126: New Node-level health check handler at /admin/info/healthcheck and /node/health paths that
checks if the node is live, connected to zookeeper and not shutdown.
(Anshum Gupta, Amrit Sarkar, shalin)
- Bug Fixes (6)
- SOLR-13058: Fix block that was synchronizing on the wrong collection in OverseerTaskProcessor
(Gus Heck)
- SOLR-11774: langid.map.individual now works together with langid.map.keepOrig. Also the detectLanguage() API
is changed to accept a Reader allowing for more memory efficient implementations
(janhoy)
- SOLR-13126: Query boosts were not being combined correctly for documents where not all boost queries
matched
(Alan Woodward, Mikhail Khludnev)
- SOLR-13248: Autoscaling based replica placement is broken out of the box. Solr 7.5 enabled autoscaling based replica
placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the
same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons,
this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until
Solr 7.4.
(Gus Heck, Andrzej Bialecki, Bram Van Dam, shalin)
- SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression
in release 7.7
(noble)
- SOLR-13299: Fix Windows startup script to disable HTTP/2 if TLS is enabled on Java 8.
(Uwe Schindler)
- Improvements (3)
- SOLR-5211: If _root_ is defined in the schema, it is now always populated automatically. This allows documents with
children to be updated with a document that does not have children, whereas before it would break block-join queries.
If you don't use nested documents then _root_ can be removed from the schema.
(Dr Oleg Savrasov, Moshe Bla,
David Smiley, Mikhail Khludnev)
- SOLR-6117: The response format has changed slightly for ReplicationHandler error-cases. All errors now have a non-200
'status' field, a 'message' field giving more details on the error, and an optional 'exception' field.
(Shalin Mangar,
Jason Gerlowski)
- SOLR-12888: The NestedUpdateProcessor (which populates internal fields for nested child docs) is now auto-registered
to run immediately prior to RunUpdateProcessor. If the schema has no _nest_*_ fields then it's a no-op.
RunUpdateProcessorFactory looks up a special/internal URP chain "_preRun_" which is implicitly defined but could be
defined by an app for customization if desired.
(David Smiley)
- Optimizations (3)
- SOLR-12725: ParseDateFieldUpdateProcessorFactory should reuse ParsePosition.
(ab)
- SOLR-13025: Due to LUCENE-8563, the BM25Similarity formula no longer includes the (k1+1) factor in the numerator
This gives a lower absolute score but doesn't affect ordering, as this is a constant factor which is the same
for every document. Use LegacyBM25SimilarityFactory if you need the old 6.x/7.x scoring. See also upgrade notes
(janhoy)
- SOLR-13130: during the ResponseBuilder.STAGE_GET_FIELDS directly copy string bytes and avoid creating String Objects
(noble)
- Other Changes (14)
- SOLR-12614: Make "Nodes" view the default in AdminUI "Cloud" tab
(janhoy)
- SOLR-12586: Upgrade ParseDateFieldUpdateProcessorFactory (present in "schemaless mode") to use Java 8's
java.time.DateTimeFormatter instead of Joda time (see upgrade notes). "Lenient" is enabled. Removed Joda Time dependency.
(David Smiley, Bar Rotstein)
- SOLR-5163: edismax now throws an exception when qf refers to a nonexistent field
(Charles Sanders, David Smiley)
- SOLR-12805: Store previous term (generation) of replica when start recovery process
(Cao Manh Dat)
- SOLR-12652: Remove SolrMetricManager.overridableRegistryName method
(Peter Somogyi via David Smiley)
- LUCENE-8513: SlowCompositeReaderWrapper now uses MultiTerms directly instead of MultiFields
(David Smiley)
- SOLR-11812: Remove backward compatibility of old LIR implementation in 8.0
(Cao Manh Dat)
- SOLR-12620: Remove the Admin UI Cloud -> Graph (Radial) view
(janhoy)
- SOLR-12775: LowerCaseTokenizer is deprecated, and should be replaced by LetterTokenizer and
LowerCaseFilter
(Alan Woodward)
- SOLR-13036: Fix retry logic in JettySolrRunner
(Gus Heck)
- SOLR-12535: Solr no longer accepts index time boosts in JSON provided to Solr.
(David Smiley)
- SOLR-13086: Improve the error message reported by DocumentObjectBinder when a setter is not found
(Gus Heck)
- SOLR-12365: Renamed class Config to XmlConfigFile
(David Smiley)
- SOLR-9515: Hadoop dependencies have been upgraded to Hadoop 3.2.0 from 2.7.2.
(Mark Miller, Kevin Risden)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.14
- Jetty 9.4.14.v20181114
- Bug fixes (10)
- SOLR-13779: Use the safe fork of simple-xml for clustering contrib.
(Dawid Weiss, Sachin Pattan)
- SOLR-13718: SPLITSHARD (async) with failures in underlying sub-operations can result in data loss
(Ishan Chattopadhyaya)
- SOLR-12291: prematurely reporting not yet finished async Collections API call as completed
when collection's replicas are colocated at least at one node
(Varun Thacker, Mikhail Khludnev)
- SOLR-13828: Improve ExecutePlanAction error handling.
- SOLR-13472: Forwarded requests should skip authorization on receiving nodes
(Ishan Chattopadhyaya)
- SOLR-13793: HttpSolrCall now maintains internal request count (_forwardedCount) for remote queries and limits them to
the number of replicas. This avoids making too many cascading calls to remote servers, which, if not restricted, can
bring down nodes containing the said collection
(Kesharee Nandan Vishwakarma, Ishan Chattopadhyaya)
- SOLR-13971: Velocity response writer's resource loading now possible only through startup parameters. Also, removed velocity
response writer from _default configset.
(Ishan Chattopadhyaya, Sachin Pattan)
- SOLR-14025: VelocityResponseWriter has been hardened - only trusted configsets can render configset provided
templates and rendering templates from request parameters has been removed.
- SOLR-13158: DataImportHandler: Added enable.dih.dataConfigParam system property to toggle whether the dataConfig param
is permitted.
(David Smiley, janhoy, Tomás Fernández Löbbe)
- SOLR-14259: Back port javabin performance regression fixes from SOLR-14013
(noble)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.14
- Jetty 9.4.14.v20181114
- Upgrade Notes (2)
- SOLR-13234: The Prometheus Metric Exporter now collects metrics from Solr at a fixed schedule that is controlled
by the exporter itself. The default interval is 60s but it can be changed by specifying the --scrape-interval
command line parameter to the tool. The collected metrics are cached and returned for any requests coming in before
the next scheduled collection. This can impact the freshness of the metrics if prometheus is configured to query the
tool more frequently.
- SOLR-12708: When requesting the status of an async request via REQUESTSTATUS collections API, the response will
include the list of internal async requests (if any) in the "success" or "failed" keys (in addition
to them being included outside those keys for backwards compatibility). See SOLR-12708 for more
details
- Bug fixes (25)
- SOLR-13234: Prometheus Metric Exporter not threadsafe. This changes the prometheus exporter to collect metrics
from Solr on a fixed interval controlled by this tool and prevents concurrent collections. This change also improves
performance slightly by using the cluster state instead of sending multiple HTTP requests to each node to lookup
all the cores.
(Danyal Prout via shalin)
- SOLR-12708: Async collection actions should not hide internal failures
(Mano Kovacs, Varun Thacker, Tomás Fernández Löbbe)
- SOLR-11876: In-place update fails when resolving from Tlog if schema has a required field
(Justin Deoliveira, janhoy,
Ishan Chattopadhyaya)
- SOLR-13349: High CPU usage in Solr due to Java 8 bug
(Erick Erickson)
- SOLR-13355: 'all' permission ignored by RuleBasedAuthorizationPlugin in most cases
(Jason Gerlowski, janhoy)
- SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin
(janhoy, Jason Gerlowski)
- SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread
(janhoy, Lorenzo)
- SOLR-13331: Atomic Update 'remove' operations broken for certain field types in SolrJ
(Thomas Wockinger via Jason Gerlowski)
- SOLR-13285: Updates with enum fields and javabin cause ClassCastException
(noble)
- SOLR-12371: Editing authorization config via REST API now works in standalone mode
(janhoy)
- SOLR-13281: Fixed NPE in DocExpirationUpdateProcessorFactory
(Munendra S N, Tomás Fernández Löbbe)
- SOLR-13409: Disable HTML directory listings in admin interface to prevent possible security issues
(Uwe Schindler)
- SOLR-13126: Query boosts were not being combined correctly for documents where not all boost queries
matched
(Alan Woodward, Mikhail Khludnev)
- SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
(Kevin Risden)
- SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used.
(ab)
- SOLR-13449: SolrClientNodeStateProvider always retries on requesting metrics from other nodes
(Cao Manh Dat)
- SOLR-13112: Upgrade jackson to 2.9.8
(Kevin Risden)
- SOLR-13410: Designated overseer wasn't able to rejoin election queue at head upon restart
(Ishan Chattopadhyaya,
Kesharee Nandan Vishwakarma)
- SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an exception
occurred while reconnecting to ZooKeeper.
(hu xiaodong via Christine Poerschke)
- SOLR-13252: Fix an NPE when setting a "policy" property for an existing collection
(ab)
- SOLR-13398: Move log line "Processing SSL Credential Provider chain..." from INFO to DEBUG
(janhoy)
- SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists
check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException.
(Mark Miller, Fernandez-Lobbe, Mike Drob)
- SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig
(Christine Poerschke)
- SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread).
(hossman)
- SOLR-13229: Cleanup replicasMetTragicEvent after all types of exception
(Tomás Fernández Löbbe)
- Other Changes (1)
- SOLR-13363: Upgrade to ZooKeeper 3.4.14
(Erick Erickson)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.13
- Jetty 9.4.14.v20181114
- Upgrade Notes (1)
- SOLR-13248: The default replica placement strategy used in Solr has been reverted to the 'legacy' policy used by Solr
7.4 and previous versions. This is due to multiple bugs in the autoscaling based replica placement strategy that was
made default in Solr 7.5 which causes multiple replicas of the same shard to be placed on the same node in addition
to the maxShardsPerNode and createNodeSet parameters being ignored. Although the default has changed, autoscaling
will continue to be used if a cluster policy or preference is specified or a collection level policy is in use.
The default replica placement strategy can be changed to use autoscaling again by setting a cluster property:
curl -X POST -H 'Content-type:application/json' --data-binary '
{
"set-obj-property": {
"defaults" : {
"cluster": {
"useLegacyReplicaAssignment":false
}
}
}
}' http://$SOLR_HOST:$SOLR_PORT/api/cluster
- Bug Fixes (2)
- SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression
in release 7.7
(noble)
- SOLR-13248: Autoscaling based replica placement is broken out of the box. Solr 7.5 enabled autoscaling based replica
placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the
same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons,
this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until
Solr 7.4.
(Gus Heck, Andrzej Bialecki, Bram Van Dam, shalin)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.13
- Jetty 9.4.14.v20181114
- Upgrade Notes (1)
- SOLR-12770: The 'shards' parameter handling logic changes to use a new config element to determine what hosts can be
requested. Please see Apache Solr Reference Guide chapter "Distributed Requests" for details, as well as SOLR-12770.
- New Features (7)
- SOLR-12839: JSON 'terms' Faceting now supports a 'prelim_sort' option to use when initially selecting
the top ranking buckets, prior to the final 'sort' option used after refinement.
(hossman)
- SOLR-7896: Add a login page to Admin UI, with initial support for Basic Auth
(janhoy)
- SOLR-13116: Add Admin UI login support for Kerberos
(janhoy, Jason Gerlowski)
- SOLR-12770: Make it possible to configure a host whitelist for distributed search
(Christine Poerschke, janhoy, Erick Erickson, Tomás Fernández Löbbe)
- SOLR-12373: Add a "tombstoneConfig" option to DocBasedVersionConstraintsProcessor that allows
users to configure which fields/values to add to tombstone documents. This can be useful to
make sure tombstone documents include fields that are marked as required in the schema
(Tomás Fernández Löbbe)
- SOLR-12984: The search Streaming Expression should properly support and push down paging
when using the /select handler
(Joel Bernstein)
- SOLR-13088: Add zplot Stream Evaluator to plot math expressions in Apache Zeppelin
(Joel Bernstein)
- Bug Fixes (18)
- SOLR-12546: CVSResponseWriter omits useDocValuesAsStored=true field when fl=*
(Munendra S N via Mikhail Khludnev)
- SOLR-12933: Fix SolrCloud distributed commit.
(Mark Miller)
- SOLR-13014: URI Too Long with large streaming expressions in SolrJ
(janhoy)
- SOLR-13066: A failure while reloading a SolrCore can result in the SolrCore not being closed.
(Mark Miller)
- SOLR-11296: Spellcheck parameters not working in new UI
(Matt Pearce via janhoy)
- SOLR-10975: New Admin UI Query does not URL-encode the query produced in the URL box
(janhoy)
- SOLR-13072: Management of markers for nodeLost / nodeAdded events is broken. This bug could have caused
some events to be lost if they coincided with an Overseer leader crash.
(ab)
- SOLR-13080: The "terms" QParser's "automaton" method semi-required that the input terms/IDs be sorted. This
query parser now does this. Unclear if this is a perf issue or actual bug.
(Daniel Lowe, David Smiley)
- SOLR-13082: A trigger that creates trigger events more frequently than the cool down period can starve other triggers.
This is mitigated to some extent by randomly choosing the trigger to resume after cool down. It is recommended that
scheduled triggers not be used for very frequent operations to avoid this problem.
(ab, shalin)
- SOLR-12514: Rule-base Authorization plugin skips authorization if querying node does not have collection replica
(noble)
- SOLR-11853: Solr installer fails on SuSE linux
(Markus Mandalka via janhoy)
- SOLR-12237: Fix incorrect SOLR_SSL_KEYSTORE_TYPE variable in solr start script
(janhoy, Joel Bernstein)
- SOLR-13053: NodeAddedTrigger and NodeLostTrigger do not reserve added/removed time populated by restoreState
(Cao Manh Dat)
- SOLR-13137: NPE when /admin/zookeeper/status endpoint hit in standalone mode
(janhoy)
- SOLR-13091: REBALANCELEADERS is broken
(Erick Erickson)
- SOLR-11998: RebalanceLeaders API broken response format with wt=JSON
(Erick Erickson)
- SOLR-9735: Fix v2 API for AutoscalingHistoryHandler.
(ab)
- SOLR-13168: Fixed a bug in TestInjection that caused test only code to be invoked when TLOG replicas
recieved commits if java assertions were enabled.
(hossman)
- Improvements (15)
- SOLR-12881: Remove unneeded import statements
(Peter Somogyi via Erick Erickson)
- SOLR-12992: When using binary format, ExportWriter to directly copy BytesRef instead of
creating new String
(noble)
- SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor.
(Mark Miller)
- SOLR-12897: Introduce AlreadyClosedException to clean up silly close / shutdown logging.
(Mark Miller)
- SOLR-12896: Introduce more checks for shutdown and closed to improve clean close and shutdown.
(Mark Miller)
- SOLR-12804: Remove static modifier from Overseer queue access.
(Mark Miller)
- SOLR-12833: Add configurable timeout to VersionBucket lock.
(Jeffery Yuan, Mark Miller)
- SOLR-12885: BinaryResponseWriter (javabin format) should directly copy from BytesRef to output
(noble)
- SOLR-12973: Admin UI "Nodes" view support for replica* replica names.
(Daniel Collins, Christine Poerschke, janhoy)
- SOLR-13090: All shipped configurations still have `maxBooleanClauses` default to 1024. But if the
`solr.max.booleanClauses` sysprop is specified, that will override the 1024 default. This enables users to
update this property across the board more easily.
(Jason Gerlowski)
- SOLR-12983: JavabinLoader should avoid creating String Objects and create UTF8CharSequence fields from byte[]
(noble)
- SOLR-13016: Computing suggestions when policy have "#EQUAL" or "#ALL" rules take too long
(noble)
- SOLR-13029: solr.hdfs.buffer.size can be configured for HdfsBackupRepository for better performance
(Tim Owen via Mikhail Khludnev)
- SOLR-13156: support facet.sort for facet.field={!terms=foo,bar}field.
(Konstantin Perikov via Mikhail Khludnev)
- SOLR-13146: Allow derivatives to be computed for the oscillate Stream Evaluator
(Joel Bernstein)
- Other Changes (7)
- SOLR-12972: deprecate unused SolrIndexConfig.luceneVersion
(Christine Poerschke)
- SOLR-12801: Make massive improvements to the tests.
(Mark Miller)
- SOLR-12923: The new AutoScaling tests are way too flaky and need special attention.
(Mark Miller)
- SOLR-12932: ant test (without badapples=false) should pass easily for developers.
(Mark Miller)
- SOLR-13036: Fix retry logic in JettySolrRunner
(Gus Heck)
- SOLR-12727: Upgrade ZooKeeper dependency to 3.4.13
(Kevin Risden, Erick Erickson, Cao Manh Dat)
- SOLR-13086: Improve the error message reported by DocumentObjectBinder when a setter is not found
(Gus Heck)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.19.1
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.11
- Jetty 9.4.11.v20180605
- Upgrade Notes (4)
- SOLR-12767: The min_rf parameter is no longer needed, Solr will always return the achieved replication factor (rf)
in the response header.
- SOLR-12827: The cluster wide defaults structure has changed from {collectionDefaults: {nrtReplicas : 2}} to
{defaults : {collection : {nrtReplicas : 2}}}. The old format continues to be supported and can be read from
ZK as well as written using the V2 set-obj-property syntax but it is deprecated and will be removed in Solr 9.
We recommend that users change their API calls to use the new format going forward.
- SOLR-12739: Autoscaling policy framework is now used as the default strategy to select the nodes on which
new replicas or replicas of new collections are created. Previously, the maxShardsPerNode parameter was not allowed
on collections when autoscaling policy was configured. Also if an autoscaling policy was configured then the default
was to set an unlimited maxShardsPerNode automatically. Now the maxShardsPerNode parameter is always
allowed during collection creation and maxShardsPerNode should be set correctly (if required) regardless of whether
autoscaling policies are in effect or not. The default value of maxShardsPerNode continues to be 1 as before. It can
be set to -1 during collection creation to fall back to the old behavior of unlimited maxShardsPerNode when using
autoscaling policy.
- SOLR-12861: Added a Solr factory for ByteBuffersDirectory, which will replace deprecated RAMDirectory in Solr 9.0.
- New Features (25)
- SOLR-6280: {!collapse}: if you attempt to use CollapseQParser on a field that is multi-valued, you will now get an
error. Previously, the collapsing behavior was unreliable and undefined despite no explicit error.
(Munendra S N, David Smiley)
- SOLR-9317: ADDREPLICA command should be able to add more than one replica to a collection,shard at a time.
The API now supports 'nrtReplicas', 'tlogReplicas', 'pullReplicas' parameters as well 'createNodeSet' parameter.
As part of this change, the CREATESHARD API now delegates placing replicas entirely to the ADDREPLICA command
and uses the new parameters to add all the replicas in one API call.
(shalin)
- SOLR-11522: /autoscaling/suggestions now include rebalance options as well even if there are no violations
(noble)
- SOLR-12822: /autoscaling/suggestions to include suggestion to add-replica for lost replicas
(noble)
- SOLR-12815: Implement maxOps limit for IndexSizeTrigger.
(ab)
- SOLR-12843: Implement a MultiContentWriter in SolrJ to post multiple files/payload at once
(noble)
- SOLR-12780: Add support for Leaky ReLU and TanH activations in contrib/ltr NeuralNetworkModel class.
(Kamuela Lau, Christine Poerschke)
- SOLR-12846: Added support for "host" variable in autoscaling policy rules
(noble)
- SOLR-5004: Splitshard collections API now supports splitting into more than 2 sub-shards directly i.e. by providing a
numSubShards parameter
(Christine Poerschke, Anshum Gupta)
- SOLR-12754: The UnifiedHighlighter has a new hl.weightMatches param defaulting to false (will be true in 8.0). It's
the highest query accuracy mode, and furthermore phrase queries are highlighted as one.
(David Smiley)
- SOLR-12828: Add oscillate Stream Evaluator to support sine wave analysis.
(Joel Bernstein)
- SOLR-11907: Add convexHull and associated geometric Stream Evaluators.
(Joel Bernstein)
- SOLR-12811: Add enclosingDisk and associated geometric Stream Evaluators.
(Joel Bernstein)
- SOLR-12840: Add pairSort Stream Evaluator
(Joel Bernstein)
- SOLR-12862: Add log10 Stream Evaluator and allow the pow Stream Evaluator to accept a vector of exponents
(Joel Bernstein)
- SOLR-12942: Add an option in IndexSizeTrigger to select the split shard method.
(ab)
- SOLR-12938: Cluster Status returns results for aliases, instead of throwing exceptions
(Gus Heck)
- SOLR-11997: Suggestions API/UI should show an entry where a violation could not be resolved
(noble)
- SOLR-12971: Add pivot Stream Evaluator to pivot facet co-occurrence counts into a matrix
(Joel Bernstein)
- SOLR-12795: Introduce 'rows' and 'offset' parameter in FacetStream
(Joel Bernstein, Amrit Sarkar, Varun Thacker)
- SOLR-11572: Add recip Stream Evaluator to support reciprocal transformations
(Joel Bernstein)
- SOLR-12936: Allow percentiles Stream Evaluator to accept an array of percentiles to calculate
(Joel bernstein)
- SOLR-12829: Add plist (parallel list) Streaming Expression
(Joel Bernstein)
- SOLR-12975: Add ltrim and rtrim Stream Evaluators
(Joel Bernstein)
- SOLR-12962: Added a new 'uninvertible' option for fields and fieldtypes. This defaults to 'true' for
backcompat allowing a FieldCache to be built for indexed fields as needed, but users are encouraged
to set this to false (using docValues as needed) to reduce the risk of large fluxuations in heap
size due to unexpected attempts to sort/facet/function on non-docValue fields.
(hossman)
- Other Changes (14)
- SOLR-12762: Fix javadoc for SolrCloudTestCase.clusterShape() method and add a method that validates only against
Active slices
(Anshum Gupta)
- SOLR-12756: Refactor Assign and extract replica placement strategies out of it. Now, assignment is done with the help
of a builder class instead of calling a method with large number of arguments. The number of special cases that had
to be handled have been cut down as well.
(shalin)
- SOLR-12827: Migrate cluster wide defaults syntax in cluster properties to a nested structure. The structure has
changed from {collectionDefaults: {nrtReplicas : 2}} to {defaults : {collection : {nrtReplicas : 2}}}.
(ab, shalin)
- SOLR-12835: Document statistics exposed by the Query Result Cache when maxRamMB is configured.
(shalin)
- SOLR-12423: Upgrade to Tika 1.19.1 when available
(Tim Allison via Erick Erickson)
- SOLR-12793: Move TestCloudJSONFacetJoinDomain and TestCloudJSONFacetSKG to the facet test package
(Varun Thacker)
- SOLR-12861: Add Solr factory for ByteBuffersDirectory.
- SOLR-12746: Simplify the Ref Guide HTML structure and use semantic HTML tags where possible. Adds new template files
for Asciidoctor HTML conversion. Building the HTML version now requires the Slim gem.
(Cassandra Targett)
- SOLR-12956: Add Javadoc @since tag to Analyzer component classes
(Alexandre Rafalovitch)
- SOLR-12966: Add Javadoc @since tag to URP classes
(Alexandre Rafalovitch)
- SOLR-12600: Fix parameter names in Solr JSON documentation
(Alexandre Rafalovitch)
- SOLR-12497: Add documentation to use Hadoop credential provider-based keystore/trustsore.
- (Mano Kovacs, Cassandra Targett)
- SOLR-13006: ZkNodeProps to be able to load from both javabin and JSON
(noble)
- Bug Fixes (20)
- SOLR-12803: Ensure ConcurrentUpdateSolrClient sends documents to correct collection
(Jason Gerlowski)
- SOLR-11836: FacetStream works with bucketSizeLimit of -1 which will fetch all the buckets.
(Alfonso Muñoz-Pomer Fuentes, Amrit Sarkar via Varun Thacker)
- SOLR-12776: Setting of TMP in solr.cmd causes invisibility of Solr to JDK tools
(Petr Bodnar via Erick Erickson)
- SOLR-12648: Autoscaling framework based replica placement is not used unless a policy is specified or
non-empty cluster policy exists.
(shalin)
- SOLR-12750: Migrate API should lock the collection instead of shard.
(shalin)
- SOLR-12767: When using min_rf, shard leader skipped bad replicas instead of marking them for recovery, this could
create inconsistencies between replicas of the same shard. min_rf parameter is now deprecated, and even if provided
replicas that don't ack an update from the leader will be marked for recovery.
(Tomás Fernández Löbbe)
- SOLR-12814: Metrics history causing "HttpParser URI is too large >8192" when many collections
(janhoy)
- SOLR-12836: ZkController creates a cloud solr client with no connection or read timeouts. Now the http client
created by the update shard handler is used instead.
(shalin)
- SOLR-12729: SplitShardCmd should lock the parent shard to prevent parallel splitting requests.
(ab)
- SOLR-12851: Improvements and fixes to let and select Streaming Expressions
(Joel Bernstein)
- SOLR-12874: Java 9+ GC Logging filesize parameter should use a unit.
(Tim Underwood via Uwe Schindler)
- SOLR-12868: Request forwarding for v2 API is broken
(noble)
- SOLR-7557: Fix parsing of child documents using queryAndStreamResponse
(Marvin Bredal Lillehaug/Stian Østerhaug via janhoy)
- SOLR-12875: fix ArrayIndexOutOfBoundsException when unique(field) or uniqueBlock(_root_) is
- used with DVHASH method in json.facet.
(Tim Underwood via Mikhail Khludnev)
- SOLR-12954: fix facet.pivot refinement bugs when using facet.sort=index and facet.mincount>1
(hossman)
- SOLR-12023: Autoscaling policy engine shuffles replicas needlessly
(noble)
- SOLR-12243: Edismax missing phrase queries when phrases contain multiterm synonyms
(Elizabeth Haubert, Alessandro Benedetti, Uwe Schindler, Steve Rowe)
- SOLR-12977: Autoscaling tries to fetch metrics from dead nodes
(noble)
- SOLR-12978: In autoscaling NPE thrown for nodes where value is absent
(noble)
- Improvements (11)
- SOLR-12767: Solr now always includes in the response of update requests the achieved replication factor
(Tomás Fernández Löbbe)
- SOLR-12782: UninvertingReader wrapping is now a bit more lightweight: Does not create FieldInfo for fields that
can't be uninverted (saves mem) and can avoid wrapping the reader altogether if there's nothing to uninvert.
IndexSchema.getUninversionMap refactored to getUninversionMapper and no longer merges FieldInfos.
(David Smiley)
- SOLR-12739: Make autoscaling policy based replica placement the default strategy for placing replicas.
(shalin)
- SOLR-12806: use autoscaling policies with strict=false to prioritize node allocation
(noble)
- SOLR-10981: Support for stream.url or stream.file pointing to gzipped data. It's detected by either a content
encoding header or file extension.
(Andrew Lundgren via David Smiley, Jan Høydahl)
- SOLR-12892: MapWriter to use CharSequence instead of String
(noble)
- SOLR-12882: Eliminate excessive lambda allocation in json facets FacetFieldProcessorByHashDV
(Tim Underwood)
- SOLR-12699: Make contrib/ltr LTRScoringModel immutable and cache its hashCode.
(Stanislav Livotov, Edward Ribeiro, Christine Poerschke)
- LUCENE-8557: Some internal LeafReader.getFieldInfos implementations were being re-computed on-demand instead of
once up front leading to some slowdowns in places like JSON Facets and field collapsing.
(Tim Underwood, David Smiley)
- SOLR-12964: Json Facets: use DocValuesIterator advanceExact() instead of advance() in FacetFieldProcessorByHashDV and
UniqueSinglevaluedSlotAcc.
(Tim Underwood)
- SOLR-12880: Json Facets: Show the FacetProcessor class name instead of the FacetRequest in the JSON Facets debug-trace
output
(Tim Underwood)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (5)
- Apache Tika 1.18
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache ZooKeeper 3.4.11
- Jetty 9.4.11.v20180605
- Upgrade Notes (6)
- The data-driven schema no longer returns the *_str field-copy of text fields by default. The *_str field is still
created and can be used for e.g. sorting, but to retrieve it you now need to explicitly list it in "fl" (SOLR-12350)
- LUCENE-7976: TieredMergePolicy now respects maxSegmentSizeMB by default when executing
findForcedMerges (optimize) and findForcedDeletesMerges (expungeDeletes)
(Erick Erickson)
- SOLR-12395: SignificantTermsQParserPlugin's name is now 'significantTerms' and its old name 'sigificantTerms' is deprecated.
- SOLR-11694: Extremely outdated UIMA contrib module has been removed
(Alexandre Rafalovitch)
- SOLR-12008: The configuration file for log4j2.xml is now in ../server/resources/log4j2.xml. All other copies have been removed.
WINDOWS USERS: This JIRA corrects a bug in the start scripts that put example logs under ...\server, solr.log is
now under ...\example.
(Erick Erickson)
- SOLR-12398: The heatmap facet response structure, when returned in JSON, changed from an even/odd name/value array to
an object style. Although the object style makes more sense, this was an overlooked back-compat break; sorry.
- New Features (42)
- SOLR-11865: The QueryElevationComponent now has a useConfiguredElevatedOrder setting. When multiple docs are elevated,
this specifies whether their relative order should be the order in the configuration file or if not then should they
be subject to whatever the sort criteria is. Additionally, QEC was extensively refactored to be more extensible.
(Bruno Roustant, David Smiley)
- SOLR-12474: Add an UpdateRequest Object that implements RequestWriter.ContentWriter
(noble)
- SOLR-12361: Allow nested child documents to be in field values of a SolrInputDocument as an alternative to
add/get ChildDocuments off to the side. The latter is now referred to as "anonymous" child documents as opposed to
"labelled" (by the field name). Anonymous child docs might be deprecated in the future. This is an internal change
that should work for javabin/SolrJ; separate issues will address XML & JSON formats populating nested docs in this way.
AddUpdateCommand and it's relationship with DirectUpdateHandler2 was reworked substantially.
(Moshe Bla, David Smiley)
- SOLR-12362: Uploading docs in JSON now supports child documents as field values, thus providing a label to the
relationship instead of the current "anonymous" relationship. Use of this experimental feature sometimes requires a
anonChildDocs=false parameter until Solr 8 due to syntax ambiguities.
(Moshe Bla, David Smiley)
- SOLR-12485: Uploading docs in XML now supports child documents as field values, thus providing a label to the
relationship instead of the current "anonymous" relationship.
(Moshe Bla, David Smiley)
- SOLR-12441: (EXPERIMENTAL) New NestedUpdateProcessorFactory (URP) to populate special fields _nest_parent_ and
_nest_path_ of nested (child) documents. It will generate a uniqueKey of nested docs if they were blank too.
(Moshe Bla, David Smiley)
- SOLR-12519: The [child] transformer now returns a nested child doc structure (attached as fields if provided this way)
provided the schema has the _nest_path_ field. This is part of a broader enhancement of nested docs.
(Moshe Bla, David Smiley)
- SOLR-12722: The [child] transformer now takes an 'fl' param to specify which fields to return. It will evaluate
doc transformers if present. In 7.5 a missing 'fl' defaults to the current behavior of all fields, but in 8.0
defaults to the top/request "fl".
(Moshe Bla, David Smiley)
- SOLR-11578: Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to give a more accurate representation
of the cluster.
(Rhoit Singh via Erick Erickson)
- SOLR-12507: Modify collection API should support un-setting properties.
(shalin)
- SOLR-12506: Add SolrJ support for the modify collection API.
(shalin)
- SOLR-12398: The JSON Facet API now supports type=heatmap facets, just as classic faceting does.
(David Smiley)
- SOLR-11985: Support percentage values in replica attribute in autoscaling policy
(noble)
- SOLR-12511: Support non integer values for replica in autoscaling policy
(noble)
- SOLR-12517: Support range values for replica in autoscaling policy
(noble)
- SOLR-12530: Ability to disable configset upload via -Dconfigset.upload.enabled=false startup parameter
(Ishan Chattopadhyaya)
- SOLR-12495: An #EQUAL function for replica in autoscaling policy to equally distribute replicas
(noble)
- SOLR-11986: Allow percentage in freedisk attribute in autoscaling policy rules
(noble)
- SOLR-12522: Support a runtime function `#ALL` for 'replica' in autoscaling policies
(noble)
- SOLR-12567: JSON Facet "functions" now support an extended "type:func" syntax, similar to other types
of facets. This also allows additional local params to be specified for if the aggregation function
can take advantage of them.
(hossman)
- SOLR-12581: the JSON Facet 'relatedness()' aggregate function now supports a 'min_popularity' option
using the extended type:func syntax
(hossman)
- SOLR-12536: autoscaling policy support to equally distribute replicas on the basis of arbitrary properties
(noble)
- SOLR-8207: Add "Nodes" view to the Admin UI "Cloud" tab, listing nodes and key metrics
(janhoy)
- SOLR-7767: "ZK Status" sub menu under "Cloud" tab to see status of zookeeper ensemble
(janhoy)
- SOLR-11990: Make it possible to co-locate replicas of multiple collections together in a node. A collection may be
co-located with another collection during collection creation time by specifying a 'withCollection' parameter. It can
also be co-located afterwards by using the modify collection API. The co-location guarantee is enforced regardless of
future cluster operations whether they are invoked manually via the Collection API or by the Autoscaling framework.
(noble, shalin)
- SOLR-12402: Factor out SolrDefaultStreamFactory class.
(Christine Poerschke)
- SOLR-12592: support #EQUAL function, range operator, decimal and percentage in cores in autoscaling policies
(noble)
- SOLR-12655: Add Korean morphological analyzer ("nori") to default distribution. This also adds examples
for configuration in Solr's schema.
(Uwe Schindler)
- SOLR-11863: Add knnRegress Stream Evaluator to support nearest neighbor regression
(Joel Bernstein)
- SOLR-12702: Add zscores Stream Evaluator
(Joel Bernstein)
- SOLR-12687: Add functions to cache data structures and mathematical models
(Joel Bernstein)
- SOLR-12671: Add robust flag to knnRegress Stream Evaluator
(Joel Bernstein)
- SOLR-12660: Add outliers Stream Evaluator to support outlier detection with probability distributions
(Joel Bernstein)
- SOLR-12634: Add gaussfit Stream Evaluator
(Joel Bernstein)
- SOLR-12629: The predict evaluator should work with the polyfit function
(Joel Bernstein)
- SOLR-12715: NodeAddedTrigger should support adding replicas to new nodes by setting preferredOperation=addreplica.
(shalin)
- SOLR-11861: When creating a configSet via the API, the "baseConfigSet" parameter now defaults to "_default".
(Amrit Sarkar, David Smiley)
- SOLR-12716: NodeLostTrigger should support deleting replicas from lost nodes by setting preferredOperation=deletenode.
(shalin)
- SOLR-9418: Added a new (experimental) PhrasesIdentificationComponent for identifying potential phrases
in query input based on overlapping shingles in the index.
(Akash Mehta, Trey Grainger, hossman)
- SOLR-11943: Add machine learning functions for location data
(Joel Bernstein)
- SOLR-12612: Cluster properties restriction of known keys only is relaxed, and now unknown properties starting with "ext."
will be allowed. This allows custom to plugins set their own cluster properties.
(Jeffery Yuan via Tomás Fernández Löbbe)
- SOLR-12357: Time Routed Aliases now have a preemptiveCreateMath option to preemptively and asynchronously create the
next collection in advance as new data gets within this time window of the end.
(Gus Heck, David Smiley)
- Bug Fixes (44)
- SOLR-12449: Response /autoscaling/diagnostics shows improper json
(noble)
- SOLR-11676: Keep nrtReplicas and replicationFactor in sync while creating a collection and modifying a collection
(Varun Thacker)
- SOLR-12489: User specified replicationFactor and maxShardsPerNode is used when specified during a restore operation.
A user can now specify nrtReplicas/tlogReplicas/pullReplicas while restoring the collection.
Specifying replicationFactor or nrtReplicas have the same effect and only one can be specified
(Varun Thacker)
- SOLR-11216: Race condition in PeerSync
(Cao Manh Dat)
- SOLR-11807: Restoring collection now treats maxShardsPerNode=-1 as unlimited
(Varun Thacker)
- SOLR-12413: If Zookeeper was pre-loaded with data before first-use, then the aliases information would be ignored.
(David Smiley, Gaël Jourdan, Gus Heck)
- SOLR-12482: Config API returns status 0 for failed operations.
(Steve Rowe)
- SOLR-12513: Reproducing TestCodecSupport.testMixedCompressionMode failure
(Erick Erickson)
- SOLR-11665: Improve error handling of shard splitting. Fix splitting of mixed replica types.
(ab)
- SOLR-12326: JSON Facet API: terms facet shard requests now indicate if they have more buckets to prevent
unnecessary refinement requests.
(yonk)
- SOLR-12427: Improve error message for invalid 'start', 'rows' parameters.
(Munendra S N via Jason Gerlowski)
- SOLR-12395: Make 'significantTerms' the SignificantTermsQParserPlugin's name and deprecate its old 'sigificantTerms' name.
(Tobias Kässmann, Christine Poerschke)
- SOLR-12533 Collection collection fails if metrics are called during core creation
(Peter Cseh, Mano Kovacs)
- SOLR-2834: Fix SolrJ Field and Document analyzes for types that include CharacterFilter
(Alexandre Rafalovitch)
- SOLR-12516: Fix some bugs in 'type:range' Facet refinement when sub-facets are combined with non
default values for the 'other' and 'include' options.
(hossman)
- SOLR-12343: Fixed a bug in JSON Faceting that could cause incorrect counts/stats when using non default
sort options. This also adds a new configurable "overrefine" option.
(Yonik Seeley, hossman)
- SOLR-12553: Allow SignificantTerms Query Parser to use local parameters
(Alexandre Rafalovitch)
- SOLR-12570: OpenNLPExtractNamedEntitiesUpdateProcessor cannot support multi fields because pattern replacement
doesn't work correctly.
(Koji Sekiguchi)
- SOLR-12576: Update ref guide for additional information displayed in cloud view
(Erick Erickson)
- SOLR-12597: Migrate API should fail requests that do not specify split.key parameter
(shalin)
- SOLR-12477: An update would return a client error(400) if it hit a AlreadyClosedException.
We now return the error as a server error(500) instead
(Jeffery via Varun Thacker)
- SOLR-12606: Fix InfixSuggestersTest.testShutdownDuringBuild() failures.
(Steve Rowe)
- SOLR-12607: Fixed two separate bugs in shard splits which can cause data loss. The first case is when using TLOG
replicas only, the updates forwarded from parent shard leader to the sub-shard leader are written only in tlog and
not the index. If this happens after the buffered updates have been replayed then the updates can never be executed
even though they remain the transaction log. The second case is when synchronously forwarding updates to sub-shard
leader fails and the underlying errors are not propagated to the client.
(Cao Manh Dat, shalin)
- SOLR-12344: SolrSlf4jReporter doesn't set MDC context.
(ab)
- SOLR-12594: MetricsHistoryHandler.getOverseerLeader fails when hostname contains hyphen.
(ab)
- SOLR-12615: HashQParserPlugin will no longer throw an NPE if the hash key field is a string when there are documents
with empty values. All documents with empty values ( string , numeric ) will be processed by worker=0
This would fix the NPE when using the search stream with partitionKeys.
(Varun Thacker)
- SOLR-11770: NPE in tvrh if no field is specified and document doesn't contain any fields with term vectors
- SOLR-12541: Metrics handler throws an error if there are transient cores.
(ab)
- SOLR-12470: Search Rate Trigger multiple bug fixes, improvements and documentation updates.
(ab)
- SOLR-12665: Autoscaling policy not being refreshed due to caching
(noble)
- SOLR-12649: CloudSolrClient retries requests unnecessarily exception from server
(noble, shalin)
- SOLR-12670: RecoveryStrategy logs wrong wait time when retrying recovery.
(shalin)
- SOLR-12668: Autoscaling trigger listeners should be executed in the order of their creation.
(ab)
- SOLR-12475: Fix MaxSizeAutoCommitTest failures
(Rupa Shankar, Anshum Gupta)
- SOLR-12674: RollupStream should not use the HashQueryParser for 1 worker.
(Varun Thacker)
- SOLR-12679: MiniSolrCloudCluster internal jetty list should never have duplicates
(shalin)
- SOLR-12598: Do not fetch non-stored fields
(Nikolay Khitrin, Erick Erickson)
- SOLR-12683: HashQuery will throw an exception if more than 4 partitionKeys is specified.
Earlier after the 4th partitionKey the keys would be silently ignored.
(Varun Thacker)
- SOLR-10028: Fix and improvements to SegmentsInfoRequestHandlerTest
(Christine Poerschke, Tomás Fernández Löbbe)
- SOLR-11585: Solr SQL does not work with point numeric fields
(Joel Bernstein, Kiran Chitturi)
- SOLR-12704: Guard AddSchemaFieldsUpdateProcessorFactory against null field names and field values.
(Steve Rowe, Varun Thacker)
- SOLR-12733: SolrMetricReporterTest failure
(Erick Erickson, David Smiley)
- SOLR-12765: Incorrect format of JMX cache stats.
(Bojan Smid, ab)
- SOLR-12749: timeseries() expression missing sum() results for empty buckets
(Joel Bernstein)
- Optimizations (12)
- SOLR-12350: Do not use docValues as stored for _str (copy)fields in _default configset
(janhoy)
- SOLR-12455: Refactor JSON serialization code into SolrJ package
(noble)
- SOLR-11654: Time Routed Alias will now route documents to the ideal shard of a collection, thus avoiding a hop.
Usually documents were already routed well but not always.
(Gus Heck, David Smiley)
- SOLR-11598: The export handler does not limit users to 4 sort fields and is now unlimited. However the speed at
which we can export is directly proportional to the number of sort fields specified. This change also allows streaming
expressions to group by on more than 4 fields.
(Aroop Ganguly, Amrit Sarkar, Varun Thacker)
- SOLR-12305: When a replica is applying updates, some kind of updates can skip buffering for faster recovery.
(Cao Manh Dat)
- SOLR-12509: Improve SplitShardCmd performance and reliability. A new method of splitting has been
introduced (splitMethod=link) which uses hard-linking of index files when possible, resulting in
significant speedups and reduced CPU / IO load on shard leader.
(ab)
- SOLR-11881: Retry update requests sent by leaders to it's followers
(Varun Thacker, Mark Miller, Tomás Fernández Löbbe)
- SOLR-12616: Optimize Export writer upto 4 sort fields to get better performance.
This was removed in SOLR-11598 but brought back in the same version
(Amrit Sarkar, Varun Thacker)
- SOLR-12572: While exporting documents using the export writer, if a field is specified as a sort parameter and also
in the fl (field list) parameter, we save on one doc-value lookup. This can bring performance improvements of 15%
and upwards depending on how many fields are in common.
(Amrit Sarkar, Varun Thacker)
- SOLR-10697: HttpShardHandler now uses a default of 100k as maxConnections (10k previously) and default
maxConnectionsPerHost as 100k (20 previously). They are now consisent with the UpdateShardHandler defaults.
(Varun Thacker)
- SOLR-12723: Reduce object creation in HashBasedRouter.
(ab)
- SOLR-12766: When retrying internal requests, backoff only once for the full batch of retries
(Tomás Fernández Löbbe)
- Other Changes (26)
- SOLR-12208: Renamed the autoscaling variable 'INDEX.sizeInBytes' to 'INDEX.sizeInGB'
(noble)
- SOLR-12523: Improve error reporting and docs regarding Collection backup feature shared-fs requirement
(janhoy)
- SOLR-12468: Upgrade Jetty to 9.4.11.v20180605
(Michael Braun, shalin)
- SOLR-12527: factor out a test-framework/ConfigRequest class
(Christine Poerschke)
- SOLR-12412: Leader should give up leadership when IndexWriter.tragedy occur
(Cao Manh Dat, Tomas Fernandez-Lobbe)
- SOLR-12551: Upgrade to Tika 1.18
(Tim Allison via Erick Erickson)
- SOLR-12464: Reduce Overseer.close() logging (for non-Overseer leaders)
(Christine Poerschke)
- SOLR-12454: Tweak Overseer leadership transition related logging for easier troubleshooting.
(Christine Poerschke)
- SOLR-12574: Put under a common "significantTerms" bucket all output by SignificantTerms Query Parser
(Alexandre Rafalovitch)
- SOLR-12164: Improve Ref Guide main landing page.
(Cassandra Targett)
- SOLR-10984: Clean up web.xml, removing old redirects and outdated comments
(Varun Thacker, janhoy)
- SOLR-12617: Remove Commons BeanUtils as a dependency
(Varun Thacker)
- SOLR-11008: Use a lighter config for MetricsHandlerTest and ensure the core is up before the test starts
(Varun Thacker)
- SOLR-11766: Move Streaming Expressions section in Ref Guide to be a top-level section.
(Cassandra Targett)
- SOLR-12656: ShardSplitTest should extend AbstractFullDistribZkTestBase instead of BasicDistributedZkTest.
(shalin)
- LUCENE-8456: Upgrade Apache Commons Compress to v1.18
(Steve Rowe)
- SOLR-12014: Cryptic error message when creating a collection with sharding that violates autoscaling policies
(noble)
- SOLR-12680: Fix ClassCastException and AIOOBE in TestSolrConfigHandlerConcurrent.
(shalin)
- SOLR-12675: Make LeaderVoteWaitTimeoutTest more resilient against side effects of test methods.
(shalin)
- SOLR-12130: CdcrReplicationDistributedZkTest is broken into two test classes, CdcrOpsAndBoundariesTest which does
not require node restarts and CdcrWithNodesRestartsTest which does. The tests themselves are made faster and more
resilient to spurious failures.
(Varun Thacker, Amrit Sarkar via shalin)
- SOLR-12625: Combine SolrDocumentFetcher and RetrieveFieldsOptimizer
(Erick Erickson)
- SOLR-12690: Regularize LoggerFactory declarations
(Erick Erickson)
- SOLR-12590: Improve Solr resource loader coverage in the ref guide.
(Steve Rowe, Cassandra Targett, Christine Poerschke)
- SOLR-12744: Improve logging messages and verbosity around recoveries
(Cao Manh Dat, Varun Thacker)
- SOLR-8742: In HdfsDirectoryTest replace RAMDirectory usages with ByteBuffersDirectory.
(hossman, Mark Miller, Andrzej Bialecki, Steve Rowe)
- SOLR-12771: Improve Autoscaling Policy and Preferences documentation.
(hossman, Steve Rowe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.17
- Carrot2 3.16.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.11
- Jetty 9.4.10.v20180503
- Upgrade Notes (3)
- SOLR-7887: Solr now uses Log4J 2.11 . The log4j configuration is now in log4j2.xml rather than log4j.properties files.
This is a server side change only and clients using SolrJ won't need any changes. Clients can still use any logging
implementation which is compatible with SLF4J. We now let log4j2 handle rotation of solr logs at startup, and bin/solr
start scripts will no longer attempt this nor move existing console or GC logs into logs/archived either (SOLR-12144).
- SOLR-11673: Slave doesn't commit empty index when completely new index is detected on master during replication.
To return the previous behavior pass false to skipCommitOnLeaderVersionZero in slave section of replication
handler configuration, or pass it to the fetchindex command.
- SOLR-11453: Configuring slowQueryThresholdMillis now logs slow requests to a separate file - solr_slow_requests.log.
Previously they would get logged in the solr.log file.
- New Features (37)
- SOLR-12396: Upgrade Carrot2 to 3.16.0, HPPC to 0.8.1, morfologik to 2.1.5.
(Dawid Weiss)
- SOLR-11200: A new CMS config option 'ioThrottle' to manually enable/disable
ConcurrentMergeSchedule.doAutoIOThrottle.
(Amrit Sarkar, Nawab Zada Asad iqbal via Dawid Weiss)
- SOLR-11670: Implement a periodic house-keeping task. This uses a scheduled autoscaling trigger and
currently performs cleanup of old inactive shards.
(ab, shalin)
- SOLR-12015: Add support "add-distinct" in AtomicURP so that we can use the 'add-distict' as a request parameter e.g:
atomic.<multival-field-name>=add-distict
(Amrit Sarkar via noble)
- SOLR-7887: Upgrade Solr to use Log4J 2.11
(Tim Potter, Keith Laban, Shawn Heisey, Ralph Goers, Erick Erickson, Varun Thacker)
- SOLR-12139: The "eq" (equals) function query now works with string fields, string literals, and perhaps anything.
(Andrey Kudryavtsev, David Smiley)
- SOLR-10783: Add support for Hadoop Credential Provider as SSL/TLS store password source.
(Mano Kovacs via Mark Miller)
- SOLR-12036: Factor out DefaultStreamFactory solrj class.
(Christine Poerschke)
- SOLR-12151: Add abstract MultiSolrCloudTestCase class.
(Christine Poerschke)
- SOLR-12181: Add index size autoscaling trigger, based on document count or size in bytes.
(ab)
- SOLR-11982: Add possibility to define replica order with the shards.preference parameter to e.g. prefer PULL replicas
for distributed queries.
(Ere Maijala, Tomás Fernández Löbbe)
- SOLR-11336: DocBasedVersionConstraintsProcessorFactory is more extensible and now supports a list of versioned fields.
(versionField config may now be a comma-delimited list).
(Michael Braun via David Smiley)
- SOLR-11913: SolrJ SolrParams now implements Iterable<Map.Entry<String, String[]>> and also has a stream() method
using it for convenience.
(David Smiley, Tapan Vaishnav)
- SOLR-11924: Added the ability to listen to changes in the set of active collections in a cloud
in the ZkStateReader, through the CloudCollectionsListener.
(Houston Putman, Dennis Gove)
- SOLR-8998: introducing uniqueBlock(_root_) aggregation as faster alternative to unique(_root_) for counting
child value facets in parents via json.facet on block index
(Dr Oleg Savrasov, Mikhail Khludnev)
- SOLR-12278: Add IgnoreLargeDocumentProcessFactory
(Cao Manh Dat, David Smiley)
- SOLR-11277: Add auto hard-commit settings based on tlog size
(Rupa Shankar, Anshum Gupta)
- SOLR-9480: A new 'relatedness()' aggregate function for JSON Faceting to enable building Semantic
Knowledge Graphs.
(Trey Grainger, hossman)
- SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate file - solr_slow_requests.log.
(Shawn Heisey, Remko Popma, Varun Thacker)
- SOLR-12401: Add getValue() and setValue() Stream Evaluators
(Joel Bernstein, janhoy)
- SOLR-12378: Support missing versionField on indexed docs in DocBasedVersionConstraintsURP.
(Oliver Bates, Michael Braun via Mark Miller)
- SOLR-12388: Enable a strict ZooKeeper-connected search request mode, in which search
requests will fail when the coordinating node can't communicate with ZooKeeper,
by setting the "shards.tolerant" param to "requireZkConnected".
(Steve Rowe)
- SOLR-9685: #Tagging queries in JSON Query DSL, equivalent to LocalParams based query/filter
tagging. Multiple tags are comma separated.
LocalParams Example : {!tag=colorfilt}color:blue
Equivalent JSON Example : { "#colorfilt" : "color:blue" }
(Dmitry Tikhonov, Mikhail Khludnev, yonik)
- SOLR-12328: JSON Facet API: Domain change with graph query.
(Daniel Meehl, Kevin Watters, yonik)
- SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate file - solr_slow_requests.log.
(Shawn Heisey, Remko Popma, Varun Thacker)
- SOLR-12401: Add getValue() and setValue() Stream Evaluators
(Joel Bernstein, janhoy)
- SOLR-11779, SOLR-12438: Basic long-term collection of aggregated metrics. Historical data is
maintained as multi-resolution time series using round-robin databases in the '.system'
collection. New /admin/metrics/history API allows retrieval of this data in numeric
or graph formats.
(ab)
- SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas, pullReplicas
(noble)
- SOLR-12389: support deeply nested json objects in clusterprops.json
(noble)
- SOLR-12376: Added the TaggerRequestHandler (AKA SolrTextTagger) for tagging text. It's used as a component of
NER/ERD systems including query-understanding. See the ref guide for more info.
(David Smiley)
- SOLR-12266: Add discrete Fourier transform Stream Evaluators
(Joel Bernstein)
- SOLR-12158: Allow the monteCarlo Stream Evaluator to support variables
(Joel Bernstein)
- SOLR-11734: Add ones and zeros Stream Evaluators
(Joel Bernstein)
- SOLR-12273: Create Stream Evaluators for distance measures
(Joel Bernstein)
- SOLR-12159: Add memset Stream Evaluator
(Joel Bernstein)
- SOLR-12221: Add valueAt Stream Evaluator
(Joel Bernstein)
- SOLR-12175: Add random field type and dynamic field to the default managed-schema
(Joel Bernstein)
- Bug Fixes (57)
- SOLR-5351: Fixed More Like This Handler to use all fields provided in mlt.fl when used with
content stream. The similarity is calculated between the content stream's value and all
fields listed in mlt.fl.
(Dawid Weiss)
- SOLR-12103: Raise CryptoKeys.DEFAULT_KEYPAIR_LENGTH from 1024 to 2048.
(Mark Miller)
- SOLR-12107: Fixed a error in [child] transformer that could ocur if documentCache was not used
(hossman)
- SOLR-12108: Fixed the fallback behavior of [raw] and [xml] transformers when an incompatble 'wt' was
specified, the field value was lost if documentCache was not used.
(hossman)
- SOLR-11551: Standardize CoreAdmin API success/failure status codes
(Jason Gerlowski, Steve Rowe)
- SOLR-12035: ExtendedDismaxQParser fails to include charfilters in nostopanalyzer
(Tim Allison via
Tomás Fernández Löbbe)
- SOLR-10734: AtomicUpdateRequestProcessor can cause wrong/old values to be set under concurrent updates for the same
document. Multithreaded test for AtomicUpdateRequestProcessor was also beefed up and fixed.
(Ishan Chattopadhyaya, Noble Paul, Amrit Sarkar, shalin)
- SOLR-11673: By default slave doesn't commit empty index when completely new index appears on master.
See Upgrade Notes to find a way to get back to the previous behavior.
(Mikhail Khludnev)
- SOLR-9399: Delete requests do not send credentials & fails for Basic Authentication
(Susheel Kumar, Aibao Luo, Nikkolay Martinov via Erick Erickson)
- SOLR-12172: Fixed race condition that could cause an invalid set of collection properties to be kept in
memory when multiple collection property changes are done in a short period of time.
(Tomás Fernández Löbbe)
- SOLR-11929: UpdateLog metrics are not initialized on core reload.
(ab, Steve Rowe)
- SOLR-11882: SolrMetric registries retained references to SolrCores when closed. A
change of SolrMetricMAnager.registerGauge and SolrMetricProducer.initializeMetrics
method signatures was required to fix it. Third party components may continue to use the old API
but should be updated to avoid this bug
(Eros Taborelli, Erick Erickson, ab)
- SOLR-12199: TestReplicationHandler.doTestRepeater(): TEST_PORT interpolation failure:
Server refused connection at: http://127.0.0.1:TEST_PORT/solr
(Mikhail Khludnev, Dawid Weiss, Steve Rowe)
- SOLR-12096: Fixed inconsistent results format of subquery transformer for distributed search (multi-shard).
(Munendra S N, Mikhail Khludnev via Ishan Chattopadhyaya)
- SOLR-12207: Just rethrowing AssertionError caused by jdk bug in reflection with invocation details.
(ab, Dawid Weiss, Mikhail Khludnev)
- SOLR-12155: Exception from UnInvertedField constructor puts threads to infinite wait.
(Andrey Kudryavtsev, Mikhail Khludnev)
- SOLR-12201: TestReplicationHandler.doTestIndexFetchOnLeaderRestart(): handle unexpected replication failures
(Steve Rowe)
- SOLR-12190: Need to properly escape output in GraphMLResponseWriter.
(yonik)
- SOLR-12214: Leader may skip publish itself as ACTIVE when its last published state is DOWN
(Cao Manh Dat)
- SOLR-12150: Fix a test bug in CdcrBidirectionalTest.testBiDir
(Steve Rowe, Amrit Sarkar via Varun Thacker)
- SOLR-10513: ConjunctionSolrSpellChecker did not work with LuceneLevenshteinDistance
(Amrit Sarkar via James Dyer)
- SOLR-11840: Fix bin/solr help-text inconsistencies
(Jason Gerlowski)
- SOLR-10169: PeerSync will hit an NPE on no response errors when looking for fingerprint.
(Erick Erickson)
- SOLR-12187: Replica should watch clusterstate and unload itself if its entry is removed
(Cao Manh Dat)
- SOLR-6286: TestReplicationHandler.doTestReplicateAfterCoreReload(): stop checking for identical
commits before/after master core reload; and make non-nightly mode test 10 docs instead of 0.
(shalin, hossman, Mark Miller, Steve Rowe)
- SOLR-9304: Fix Solr's HTTP handling to respect '-Dsolr.ssl.checkPeerName=false' aka SOLR_SSL_CHECK_PEER_NAME
(Shawn Heisey, Carlton Findley, Robby Pond, hossman)
- SOLR-12250: NegativeArraySizeException on TransactionLog if previous document more than 1.9GB
(Cao Manh Dat)
- SOLR-12253: Remove optimize button from the core admin page
(Erick Erickson)
- SOLR-11833: Allow searchRate trigger to delete replicas. Improve configurability of the trigger by specifying
upper / lower thresholds and respective actions
(ab)
- SOLR-12261: If you attempt to delete a collection immediately after deleting an alias it used to be a part of, you may
get an error that's it's a member of that alias. This check now ensures the alias state is sync()'ed with ZK first.
(David Smiley)
- SOLR-12275: wrong caching for {!filters} as well as for `filters` local param in {!parent} and {!child}
(David Smiley, Mikhail Khluldnev)
- SOLR-12284: WordBreakSolrSpellchecker will no longer add parenthesis in collations when breaking words in
non-boolean queries.
(James Dyer)
- SOLR-12290: Do not close any servlet streams and improve our servlet stream closing prevention code for users
and devs.
(Mark Miller, janhoy, Andrzej Bialecki)
- SOLR-12293: Updates need to use their own connection pool to maintain connection reuse and prevent spurious
recoveries.
(Mark Miller)
- SOLR-12308: LISTALIASES is now assured to return an up-to-date response. Also, the test utility
MiniSolrCloudCluster.deleteAllCollections will now first delete aliases since a collection cannot be deleted if an
alias refers to it.
(David Smiley)
- SOLR-12192: Fixed a solr cli error message when ulimits are unlimited.
(Martijn Koster)
- SOLR-12258: A V2 request referencing a collection or alias may fail to resolve it if it was just recently created.
Now we sync with ZooKeeper and try one more time. V1 partially did this but only for aliases; now it does both.
(David Smiley)
- SOLR-12170: JSON Facet API: Terms facet on a date field sometimes failed with an exception complaining
about "Invalid Date String".
(yonik)
- SOLR-12307: exiting OverseerTriggerThread without endless noise in log when Zookeeper session is expired
(Mikhail Khludnev)
- SOLR-12200: abandon OverseerExitThread when ZkController is closed.
(Mikhail Khludnev)
- SOLR-12355: Fixes hash conflict in HashJoinStream and OuterHashJoinStream
(Dennis Gove)
- SOLR-12377: Don't spin off overseer when Zk controller is closed
(Mikhail Khludnev)
- SOLR-3567: Spellcheck custom parameters not being passed through due to wrong prefix creation.
(Josh Lucas via shalin)
- SOLR-12358: Autoscaling suggestions fail randomly with sorting
(noble)
- SOLR-12294: update processors loaded from runtime jars fail to load if they are specified
in an update processor chain
(noble)
- SOLR-12314: Use http timeout's defined in solr.xml for creating ConcurrentUpdateSolrClient during
indexing requests between leader and replica
( Mark Miller, Varun Thacker)
- SOLR-12290: Do not close any servlet streams and improve our servlet stream closing prevention code for users
and devs.
(Mark Miller)
- SOLR-12374: SnapShooter.getIndexCommit can forget to decref the searcher; though it's not clear in practice when.
(David Smiley)
- SOLR-12417: velocity response writer should enforce valid function name for v.json parameter
(Mano Kovacs, yonik)
- SOLR-12271: Fixed bug in how Analytics component reads negative values from float and double fields.
(Houston Putman)
- SOLR-12433: Recovering flag of a replica is set equals to leader even it failed to receive update
on recovering.
(Cao Manh Dat)
- SOLR-12354: Register the /admin/info/key end-point at the startup time to avoid 404
(noble)
- SOLR-12445: Upgrade Dropwizard Metrics to version 3.2.6.
(ab)
- SOLR-12434: bin/solr {config,healthcheck} ignore ZK_HOST in solr.in.{sh,cmd}
(Steve Rowe)
- SOLR-12481: update.autoCreateFields must be set via Config API command 'set-user-property',
but 'bin/solr create' tells users to use the default action 'set-property', which fails
because the property is not editable.
(Steve Rowe)
- SOLR-12416: When creating a time routed alias, the router.autoDeleteAge option wasn't considered.
(Joachim Sauer via David Smiley)
- SOLR-12450: Don't allow referal to external resources in various config files (CVE-2018-8026).
(Yuyang Xiao, Uwe Schindler)
- Optimizations (13)
- SOLR-11920: IndexFetcher now fetches only those files (from master/leader) that are different. This
differential fetching now speeds up recovery times when full index replication is needed, but only
a few segments diverge.
(Ishan Chattopadhyaya, Shaun Sabo, John Gallagher)
- SOLR-11731: LatLonPointSpatialField can now decode points from docValues when stored=false docValues=true,
albeit with maximum precision of 1.37cm
(Karthik Ramachandran, David Smiley)
- SOLR-11891: DocStreamer now respects the ReturnFields when populating a SolrDocument, reducing the
number of unneccessary fields a ResponseWriter will see if documentCache is used
(wei wang, hossman)
- SOLR-12312: Replication IndexFetcher should cap its internal buffer size when the file being transferred is small.
(Jeff Miller, David Smiley)
- SOLR-12333: Removed redundant lines for handling lists in JSON reponse writers.
(David Smiley via Mikhail Khludnev)
- SOLR-11880: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed
search and update operations.
(Varun Thacker, shalin)
- SOLR-12375: Optimize Lucene ScoreMode use:
A non-cached filter query could be told incorrectly that scores were needed.
The /export (ExportQParserPlugin) would declare incorrectly that scores are needed.
Expanded docs (expand component) could be told incorrectly that scores are needed.
(David Smiley)
- SOLR-12338: Replay buffering tlog in parallel.
(Cao Manh Dat, David Smiley)
- SOLR-12366: A slow "live docs" implementation was being used instead of a bitset. Affects classic faceting
enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser.
(David Smiley)
- SOLR-12337: Remove the obsolete QueryWrapperFilter intermediate wrapper, which also removed some needless uses of
SolrConstantScoreQuery as well. QWF since v5.4.0 sometimes needlessly internally executed and cached the query.
Affects ExpandComponent, ChildDocTransformer, CurrencyFieldType, TermsQParser.
(David Smiley)
- SOLR-9922: Write buffering updates to another tlog.
(Cao Manh Dat)
- SOLR-12233: QParserPlugin's built-in static registry now holds actual QParserPlugin instances instead of class
references. This is consistent with other plugin registries and allows a SolrCore to load faster.
(Jeff Miller, David Smiley)
- SOLR-12198: Stream Evaluators should not copy matrices needlessly
(Joel Bernstein)
- Other Changes (29)
- SOLR-12018: Remove comments.apache.org integration for the Ref Guide; the comments system has been down since December 2017 and there is no concrete plan to bring it back.
(Cassandra Targett)
- SOLR-12076: Remove unnecessary printLayout usage in CDCR tests
(Varun Thacker)
- SOLR-12086: Fix format problem in FastLRUCache description string shown on Cache Statistics page.
(Sathiya N Sundararajan via shalin)
- SOLR-12090: Move DistribStateManager, NodeStateProvider and SolrCloudManager interfaces out of the
autoscaling package.
(shalin)
- SOLR-12091: Rename TimeSource.getTime to getTimeNs.
(ab)
- SOLR-12101: ZkTestServer was not handling connection timeout settings properly.
(Gus Heck via Mark Miller)
- SOLR-11331: Ability to run and debug standalone Solr and a single node SolrCloud server from Eclipse.
Also being able to run all Lucene and Solr tests as a configuration
(Karthik Ramachandran via
Varun Thacker, Uwe Schindler)
- SOLR-12118: Solr Ref-Guide can now use some ivy version props directly as attributes in content
(hossman)
- SOLR-12152: Split up TriggerIntegrationTest into multiple tests to isolate and increase reliability.
(shalin)
- SOLR-12133: Fix race conditions that caused TriggerIntegrationTest.testEventQueue and testNodeMarkersRegistration
to fail.
(Mark Miller, shalin)
- SOLR-12169: Fix ComputePlanActionTest.testSelectedCollections fails on jenkins by aggressively cleaning up
trigger state left by other test methods in the test setup.
(shalin)
- SOLR-12144: SOLR_LOG_PRESTART_ROTATION now defaults to false, we leverage log4j2 for log rotation on startup
(janhoy)
- SOLR-12095: AutoScalingHandler validates trigger configurations before updating Zookeeper.
(ab)
- SOLR-12165: Ref Guide: DisMax default mm param value is improperly documented as 100%.
(Steve Rowe)
- SOLR-12154: Disallow explicit usage of Log4j2 logger via forbidden APIs.
(Varun Thacker, Tomás Fernández Löbbe)
- SOLR-12176: Improve FORCELEADER to handle the case when a replica win the election but does not present
in clusterstate
(Cao Manh Dat)
- SOLR-12134: ref-guide 'bare-bones html' validation is now part of 'ant documentation' and validates
javadoc links locally.
(hossman)
- SOLR-12142: EmbeddedSolrServer should use req.getContentWriter
(noble)
- SOLR-12252: Fix minor compiler and intellij warnings in autoscaling policy framework.
(shalin)
- SOLR-11914: The following SolrParams methods are now deprecated: toSolrParams (use NamedList.toSolrParams instead),
toMap, toMultiMap, toFilteredSolrParams, getAll. The latter ones have no direct replacement but are easy to
implement yourself as-needed.
(David Smiley)
- SOLR-10036: Upgrade jackson from 2.5.4 to 2.9.5
(Shashank Pedamallu, Shawn Heisey, Varun Thacker)
- SOLR-12289: Add more MDC logging information to collection admin requests
(Varun Thacker)
- SOLR-12288: Add more MDC logging information to core admin requests
(Varun Thacker)
- SOLR-12265: Upgrade Jetty to 9.4.10.v20180503
(Varun Thacker, Steve Rowe)
- SOLR-12374: Added SolrCore.withSearcher(lambda) to make grabbing the searcher easier than the more awkward
RefCounted API.
(David Smiley)
- SOLR-12183: Refactor Streaming Expression test cases
(Joel Bernstein)
- SOLR-12437: Document 'bin/solr config' in the ref guide.
(Steve Rowe)
- SOLR-12435: Fix bin/solr help and ref guide text to describe ZK_HOST in solr.in.sh/solr.in.cmd
as an alternative to -z cmdline param.
(Steve Rowe)
- SOLR-12428: Solr LTR jar now included in _default configset's solrconfig.xml
(Ishan Chattopadhyaya)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.17
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.11
- Jetty 9.4.8.v20171121
- Bug Fixes (9)
- SOLR-12204: Upgrade commons-fileupload dependency to 1.3.3 to address CVE-2016-1000031.
(Steve Rowe)
- SOLR-12256: Fixed some eventual-consistency issues with collection aliases by using ZooKeeper.sync().
(David Smiley)
- SOLR-12087: Deleting replicas sometimes fails and causes the replicas to exist in the down
state
(Cao Manh Dat)
- SOLR-12146: LIR should skip deleted replicas
(Cao Manh Dat)
- SOLR-12066: Cleanup deleted core when node start
(Cao Manh Dat)
- SOLR-12065: A successful restore collection should mark the shard state as active and not buffering
(Rohit, Varun Thacker)
- SOLR-11724: Cdcr bootstrapping should ensure that non-leader replicas should sync with the leader
(Amrit Sarkar, Varun Thacker)
- SOLR-12202: Fix errors in solr-exporter.cmd.
(Minoru Osuka via koji)
- SOLR-12316: Do not allow to use absolute URIs for including other files in solrconfig.xml
and schema parsing (CVE-2018-8010).
(Ananthesh, Ishan Chattopadhyaya, Uwe Schindler)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.16
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.11
- Jetty 9.4.8.v20171121
- Upgrade Notes (10)
- SOLR-11748: The throttling mechanism used to limit the rate of autoscaling events processed
has been removed. This deprecates the 'actionThrottlePeriodSeconds' setting in the set-properties
Autoscaling API which is now a no-op. Use the 'triggerCooldownPeriodSeconds' instead to pause event
processing.
- SOLR-11798: The top-level <highlighting> syntax in solrconfig.xml is now formally
deprecated in favour of <searchComponent> equivalent syntax. See also SOLR-1696.
- SOLR-11809: QueryComponent's rq parameter parsing no longer considers the defType parameter.
- SOLR-11747: The behaviour of the autoscaling system has been modified to pause all triggers from execution between
the start of actions and end of cool down period. The triggers will be resumed after the cool down period expires.
Previously, the cool down period was a fixed period started after actions for a trigger event complete and during
this time, all triggers continued to run but any events were rejected to be tried later.
- SOLR-11624: Collections created without specifying a configset name use a copy of the _default configset since 7.0.
Before 7.3, the copied over configset was named the same as the collection name, but 7.3 onwards it will be named
with an additional ".AUTOCREATED" suffix.
- SOLR-11702: The old Leader-In-Recovery implementation (SOLR-5495) is now deprecated and replaced. Solr will support
rolling upgrades from old 7.x versions of Solr to future 7.x releases until the last release of
the 7.x major version. This means in order to upgrade to Solr 8 in the future, you must be on Solr 7.3 or higher.
- LUCENE-8161: If you are using the spatial JTS library with Solr, you must upgrade to 1.15.0. This new version
of JTS is now dual-licensed to include a BSD style license.
- SOLR-12051: A new mechanism is introduced in SOLR-11702 to maintain consistency in SolrCloud between leader and replicas.
This mechanism lets Solr know whether a replica is in-sync with the leader or not, even when the leader is not live.
If all the replicas who participate in the leader election are out-of-sync with previous leader, the election will
pause until a timeout (named "leaderVoteWait") before allowing an out-of-sync replica to become leader. Note that the
new leader still needs to contains more updates than any other active replicas in the same shard. Therefore by
increasing leaderVoteWait will increase the consistency (over availability) of the system. The default value of
leaderVoteWait is 180,000 ms (3 minutes) and it can be adjusted in the "solrcloud" section of the solr.xml
- SOLR-11957: The default Solr log file size and number of backups is raised to 32MB and 10 respectively
- SOLR-12067: The default value of `autoReplicaFailoverWaitAfterExpiration` has been increased to 120 seconds
from the earlier default of 30 seconds. This affects how soon Solr adds new replicas to replace the replicas
on nodes which have either crashed or shutdown.
- New Features (44)
- SOLR-11285: Simulation framework for autoscaling.
(ab)
- LUCENE-2899: In the Solr analysis-extras contrib, added support for the
OpenNLP-based analysis components in the Lucene analysis/opennlp module:
tokenization, part-of-speech tagging, phrase chunking, and lemmatization.
Also added OpenNLP-based named entity extraction as a Solr update request
processor.
(Lance Norskog, Grant Ingersoll, Joern Kottmann, Em, Kai Gülzau,
Rene Nederhand, Robert Muir, Steven Bower, Steve Rowe)
- SOLR-11201: Implement autoscaling trigger for arbitrary metrics that creates events when
a given metric breaches a threshold
(shalin)
- SOLR-11653: TimeRoutedAlias URP now auto-creates new collections on the fly according to alias metadata
rules that sets the time interval for each collection. An internal Overseer command "ROUTEDALIAS_CREATECOLL"
was created to facilitate this.
(David Smiley)
- SOLR-11062: new tag "diskType" in autoscaling policy
(noble)
- SOLR-11063: Suggesters should accept required freedisk as a hint
(noble)
- SOLR-3218: Added range faceting support for CurrencyFieldType. This includes both "facet.range" as well
as json.facet's "type:range"
(Andrew Morrison, Jan Høydahl, Vitaliy Zhovtyuk, hossman)
- SOLR-11064: Collection APIs should use the disk space hint when using policy framework
(noble)
- SOLR-11854: multivalued primitive fields can now be sorted by implicitly choosing the min/max
value for asc/desc sort orders.
(hossman)
- SOLR-11592: Add OpenNLP language detection to the langid contrib.
(Koji, Steve Rowe)
- SOLR-11648: A new admin UI to display and execute suggestions
(Apoorv Bhawsar , noble)
- SOLR-11722: Added "time routed alias" creation support to the CREATEALIAS command. It's for managing multiple
collections partitioned by time.
(Gus Heck, David Smiley)
- SOLR-11782: Refactor LatchWatcher.await to protect against spurious wakeup
(Tomás Fernández Löbbe, David Smiley, Dawid Weiss)
- SOLR-11617: Alias properties (formerly "metadata") are now mutable via a new ALIASPROP command.
These properties are returned from LISTALIASES.
(Gus Heck via David Smiley)
- SOLR-11702: Redesign current LIR implementation
(Cao Manh Dat, shalin)
- SOLR-11376: Support computing plans for only specific collections.
(ab)
- SOLR-11681: Add ttest and pairedTtest Stream Evaluators
(Joel Bernstein)
- SOLR-11785: Add multiVariateNormalDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11789: Add integrate Stream Evaluator to support integral calculations
(Joel Bernstein)
- SOLR-11791: Add density Stream Evaluator
(Joel Bernstein)
- SOLR-11172: Add Mann-Whitney U test Stream Evaluator
(Mathew Skaria, Joel Bernstein)
- SOLR-11808: Add sumSq Stream Evaluator
(Joel Bernstein)
- SOLR-11430: Add lerp and akima Stream Evaluators to support linear and akima spline interpolation
(Joel Bernstein)
- SOLR-11683: Add chiSquareDataSet Stream Evaluator
(Joel Bernstein)
- SOLR-10716: Add termVectors Stream Evaluator
(Joel Bernstein)
- SOLR-11737: Add kmeans Stream Evaluator to support kmeans clustering
(Joel Bernstein)
- SOLR-11736: Rename knn Streaming Expression to knnSearch and add new knn Stream Evaluator
(Joel Bernstein)
- SOLR-11867: Add indexOf, rowCount and columnCount StreamEvaluators
(Joel Bernstein)
- SOLR-11862: Add fuzzyKmeans Stream Evaluatory
(Joel Bernstein)
- SOLR-11890: Add multiKmeans Stream Evaluator
(Joel Bernstein)
- SOLR-11916: new SortableTextField which supports analysis/searching just like TextField, but also
sorting/faceting just like StrField. By default uses only the first 1024 chars of the original
input string values, but this is configurable.
(hossman)
- SOLR-11778: Add per-stage RequestHandler metrics.
(ab)
- SOLR-11925: Time Routed Aliases can have their oldest collections automatically deleted via the "router.autoDeleteAge"
setting.
(David Smiley)
- SOLR-11941: Add abstract contrib/ltr AdapterModel to facilitate the development of scoring models that delegate
scoring to an opaque pre-trained model.
(Christine Poerschke)
- SOLR-11689: Add l1norm, l2norm and linfnorm Stream Evaluators
(Joel Bernstein)
- SOLR-11588: Add matrixMult Stream Evaluator to support matrix multiplication
(Joel Bernstein)
- SOLR-12006: Add a '*_t' and '*_t_sort' dynamic field for single valued text fields
(Varun Thacker)
- SOLR-11597: Add contrib/ltr NeuralNetworkModel class.
(Michael A. Alcorn, Yuki Yano, Christine Poerschke)
- SOLR-11066: Implement a scheduled autoscaling trigger that runs on a fixed interval beginning with a
given start time.
(David Smiley, Gus Heck, ab, shalin)
- SOLR-11795: Add Solr metrics exporter for Prometheus
(Minoru Osuka via koji)
- SOLR-11267: Add support for "add-distinct" atomic update operation
(Amrit Sarkar via noble )
- SOLR-11960: Add collection level properties similar to cluster properties
(Peter Rusko, Tomás Fernández Löbbe)
- SOLR-12077: Add support for autoAddReplicas in the collection creation dialog in Admin UI.
(shalin)
- SOLR-9510: introducing {!filters param=$fq excludeTags=f} query parser.
Introducing {!.. filters=$fq excludeTags=t,q} in {!parent} and {!child}
(Dr. Oleg Savrasov via Mikhail Khludnev)
- Bug Fixes (35)
- SOLR-11783: Rename core in solr standalone mode is not persisted
(Erick Erickson)
- SOLR-11555: If the query terms reduce to nothing, filter(clause) produces an NPE whereas
fq=clause does not
(Erick Erickson)
- SOLR-11824: Fixed bucket ordering in distributed json.facet type:range when mincount>0
(hossman)
- SOLR-11821: ConcurrentModificationException in SimSolrCloudTestCase.tearDown
(shalin)
- SOLR-11631: The Schema API should return non-zero status when there are failures.
(Noble Paul, Steve Rowe)
- SOLR-11839: Fix test failures resulting from SOLR-11218
(Erick Erickson)
- SOLR-11794: PULL replicas stop replicating after collection RELOAD
(Samuel Tatipamula, Tomás Fernández Löbbe)
- SOLR-11714: AddReplicaSuggester / ComputePlanAction infinite loop.
(ab)
- SOLR-11895: Logging Screen in the Admin UI will now show "No Events available" when there are no events to show.
Also, the spinner icon is removed to prevent users thinking the page has hung.
(Cassandra Targett, Shawn Heisey)
- SOLR-10525: Stacked recovery requests do no cancel an in progress recovery first.
(Mike Drob via Cao Manh Dat)
- SOLR-11873: Use time based expiration cache in all necessary places in HdfsDirectoryFactory.
(Mihaly Toth via Mark Miller)
- SOLR-11661: New HDFS collection reuses unremoved data from a deleted HDFS collection with same name causes
inconsistent view of documents
(Cao Manh Dat, shalin)
- SOLR-11459: In-place update of nonexistent doc following existing doc update fails to create the doc.
(Andrey Kudryavtsev via Mikhail Khludnev)
- SOLR-11931: Fix contrib/ltr custom inner class feature/normaliser/model persistence.
(Christine Poerschke)
- SOLR-10261: In case of in-place updates, failure in leader to follower replica update request now throws the
follower replica in leader-initiated-recovery
(Ishan Chattopadhyaya, Steve Rowe)
- SOLR-11898: ConcurrentModificationException when calling org.apache.solr.core.SolrInfoBean.getMetricsSnapshot
(Jeff Miller via Erick Erickson)
- SOLR-11950: Allow CLUSTERSTATUS "shard" parameter to accept comma (,) delimited list
(Chris Ulicny via Jason Gerlowski)
- SOLR-11739: Fix race condition that made Solr accept duplicate async IDs in collection API operations
(Tomás Fernánadez Löbbe)
- SOLR-11988: Fix exists() method in EphemeralDirectoryFactory/MockDirectoryFactory to prevent false positives
(hossman)
- SOLR-11971: Don't allow referal to external resources in DataImportHandler's dataConfig request parameter (CVE-2018-1308).
(麦 香浓郁, Uwe Schindler)
- SOLR-12021: Fixed a bug in ApiSpec and other JSON resource loading that was causing unclosed file handles
(hossman)
- SOLR-10720: Aggressive removal of a collection breaks cluster status API.
(Alexey Serba, shalin)
- SOLR-12050: UTILIZENODE does not enforce policy rules
(hossman, noble)
- SOLR-11843: Admin UI collection creation was not properly handling the router.field and router.name parameters.
Also changed the display label in the Admin UI from routerField to router.field to match the actual API.
(Shawn Heisey via Cassandra Targett)
- SOLR-12011: Consistence problem when in-sync replicas are DOWN.
(Cao Manh Dat)
- SOLR-12020: JSON Facet API: terms facet on date field fails in refinement phase with
"Invalid Date String" error.
(yonik)
- SOLR-12061: Fix substitution bug in API V1 to V2 migration when using SolrJ with V2 API.
(Tomás Fernánadez Löbbe)
- SOLR-11976: TokenizerChain.normalize: only the first filter that is a MultiTermAwareComponent was participating
in normalization instead of all. This bug normally doesn't matter since TextField doesn't call it.
(Tim Allison via David Smiley)
- SOLR-12072: Invalid path string using ZkConfigManager.copyConfigDir(String fromConfig, String toConfig)
(Alessandro Hoss via Erick Erickson)
- SOLR-12064: JSON Facet API: fix bug where a limit of -1 in conjunction with multiple facets or
missing=true caused an NPE or AIOOBE.
(Karthik Ramachandran via yonik)
- SOLR-12083: Fix RealTime GET to work on a cluster running CDCR when using Solr's in-place updates
(Amrit Sarkar, Varun Thacker)
- SOLR-12063: Fix PeerSync, Leader Election failures and CDCR checkpoint inconsistencies on a cluster running CDCR
(Amrit Sarkar, Varun Thacker)
- SOLR-12110: Replica which failed to register in Zk can become leader
(Cao Manh Dat)
- SOLR-12129: After the core is reloaded, term of the core will not be watched
(Cao Manh Dat)
- SOLR-12141: Fix "bin/solr" shell scripts (Windows/Linux/Mac) to correctly detect major Java version
and use numerical version comparison to enforce minimum requirements. Also remove obsolete "UseParNewGC" option.
This allows to start Solr with Java 10 or later.
(Uwe Schindler)
- Optimizations (5)
- SOLR-11711: Fixed distributed processing of facet.field/facet.pivot sub requests to prevent requesting
unneccessary and excessive '0' count terms from each shard
(Houston Putman via hossman)
- LUCENE-8149: NRTCachingDirectory does not need to preemptively delete segment files and generate exceptions
(Erick Erickson)
- SOLR-11879: avoid EOFException for empty input streams
(noble)
- SOLR-8327: Cluster state caching to avoid live fetch from ZK of cluster state for forwarding requests on nodes
that do not host any replica of the collection
(Jessica Cheng Mallet, John Gallagher, Noble Paul, Ishan Chattopadhyaya)
- SOLR-11769: Optimize when useFilterForSortedQuery=true and there are no filter queries by avoiding
needless match-all-docs bitset construction.
(Betim Deva, David Smiley)
- Other Changes (45)
- SOLR-11629: Add more intuitive CloudSolrClient.Builder constructors
(Varun Thacker, Jason Gerlowski)
- SOLR-11575: Improve ref-guide solrj snippets via mock 'print()' method
(Jason Gerlowski via hossman)
- SOLR-11757: In tests, fix race condition on SolrException.ignoreException.
Also ensure we register "ignore_exception" in @BeforeClass (previously only @AfterClass)
(David Smiley)
- SOLR-11754: Remove AbstractSolrTestCase which has long been supplanted by SolrTestCaseJ4.
(David Smiley)
- SOLR-11701: Upgrade to Tika 1.17 when available
(Tim Allison, Karthik Ramachandran via Erick Erickson)
- SOLR-11703: Solr Should Send Log Notifications if Ulimits are too low
(Kevin Cowan via Erick Eickson)
- SOLR-11793: Reduce code duplication w.r.t. RestTestHarness(es).
(Christine Poerschke)
- SOLR-7733: remove "optimize" from the UI.
(Erick Erickson)
- SOLR-11748: Remove Autoscaling action throttle.
(shalin)
- SOLR-11805: SolrJ's SolrResponse.getElaspedTime was sometimes a millisecond off.
(David Smiley)
- SOLR-11798: Formally deprecate top-level <highlighting> syntax in solrconfig.xml
in favour of <searchComponent> equivalent syntax.
(Christine Poerschke)
- SOLR-11801: Support customisation of the "highlighting" query response element.
(Ramsey Haddad, Pranav Murugappan, David Smiley, Christine Poerschke)
- SOLR-11692: SolrDispatchFilter's use of a "close shield" in tests should not be applied to
further servlet chain processing.
(Jeff Miller, David Smiley)
- SOLR-11218: Fail and return an error when attempting to delete a collection that's part of an alias
(Erick Erickson)
- SOLR-11817: Move Collections API classes to it's own package
(Varun Thacker)
- SOLR-11810: Upgrade Jetty to 9.4.8.v20171121
(Varun Thacker, Erick Erickson)
- SOLR-11747: Pause triggers until actions finish executing and the cool down period expires.
(shalin)
- SOLR-11871: MoveReplicaSuggester should not suggest leader if other replicas are available
(noble)
- SOLR-11624: Collections created from _default configset will now be associated with a configset with a suffix
.AUTOCREATED. For example, a new collection "mycollection", created without specifying a configset name, will
use the _default configset and the associated configset name will be "mycollection.AUTOCREATED". If this
collection is deleted and re-created, the autocreated configset will be left behind and will be re-used for
the re-created collection
(Ishan Chattopadhyaya, Abhishek Kumar Singh)
- SOLR-11051: Use disk free metric in default cluster preferences
(noble)
- SOLR-11658: Upgrade ZooKeeper dependency to 3.4.11
(Jason Gerlowski, Steve Rowe, Erick Erickson)
- SOLR-11480: Remove unused "Admin Extra" files and mentions.
(Eric Pugh, Christine Poerschke)
- SOLR-11067: REPLACENODE should identify appropriate nodes if targetNode is not provided
(noble)
- SOLR-11848: Update Ref Guide to include info on grouping operations and using curl for large files.
(Dariusz Wojtas
via Cassandra Targett)
- SOLR-11613: Make message for missing dataimport config in UI more explicit.
(Shawn Heisey, Amrit Sarkar via
Cassandra Targett)
- SOLR-11933: Make DIH UI page safer by not default checking the clean checkbox
(Eric Pugh via Tomás Fernández Löbbe)
- SOLR-11349: Rename ResponseBuilder's getQueryCommand to createQueryCommand.
(Christine Poerschke)
- SOLR-11902: Clarify in bin/solr help text whether commands can be run remotely
(Jason Gerlowski)
- SOLR-3089: RequestBuilder now exposes isDistrib() method. Using this, plugins can know whether a request is
distributed.
(Rok Rejc, Frank Wesemann, Abhishek Kumar Singh, Mikhail Khludnev via Ishan Chattopadhyaya)
- SOLR-8090: Make text elements in the UI darker for better contrast and readability
(Cassandra Targett)
- SOLR-6057: Change highlight color in UI Analysis screen
(Cassandra Targett)
- SOLR-11897: Add toggle to disable auto-refresh of Logging page in the UI
(Tommy Marshment-Howell via
Cassandra Targett)
- SOLR-12017: Remove BadApple and AwaitsFix annotations that link to closed JIRAs
(Erick Erickson)
- SOLR-12027: Increase thread lingering timeout to 80s.
(Mikhail Khludnev)
- SOLR-10809: Get precommit lint warnings out of Solr core
(Erick Erickson)
- SOLR-12028: BadApple and AwaitsFix annotations usage
(Erick Erickson, Uwe Schindler)
- SOLR-12031: Refactor Policy framework to make simulated changes affect more than a single node
(noble)
- SOLR-11957: Increase MaxFileSize=32MB and MaxBackupIndex=10 for RollingFileAppender in log4j.properties
(Varun Thacker, shalin)
- SOLR-12047: Increase checkStateInZk timeout
(Cao Manh Dat, Varun Thacker)
- SOLR-12051: Election timeout when no replicas are qualified to become leader
(Cao Manh Dat)
- SOLR-12067: Increase autoAddReplicas default 30 second wait time to 120 seconds.
(Varun Thacker, Mark Miller via shalin)
- SOLR-12078: Fixed reproducable Failure in TestReplicationHandler.doTestIndexFetchOnLeaderRestart that happened
due to using stale http connections.
(Gus Heck, shalin)
- SOLR-12099: Remove reopenReaders attribute from 'IndexConfig in SolrConfig' page in ref guide.
(shalin)
- SOLR-12098: Document the Lucene spins auto-detection and its effect on CMS dynamic defaults.
(Cassandra Targett, shalin)
- SOLR-12097: Document the diskType policy attribute and usage of disk space in Collection APIs.
(Cassandra Targett, shalin)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.16
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.20.v20170531
- Bug Fixes (4)
- SOLR-11771: Overseer can never process some last messages
(Cao Manh Dat)
- SOLR-11783: Rename core in solr standalone mode is not persisted
(Erick Erickson)
- SOLR-11809: QueryComponent.prepare rq parsing could fail under SOLR 7.2.0 - fix:
QueryComponent's rq parameter parsing no longer considers the defType parameter.
(Christine Poerschke and David Smiley in response to bug report/analysis
from Dariusz Wojtas and Diego Ceccarelli)
- SOLR-11555: If the query terms reduce to nothing, filter(clause) produces an NPE whereas
fq=clause does not
(Erick Erickson)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.16
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.20.v20170531
- Upgrade Notes (2)
- SOLR-11501: Starting a query string with local-params {!myparser ...} is used to switch the query parser to another,
and is intended for use by Solr system developers, not end users doing searches. To reduce negative side-effects of
unintended hack-ability, we've limited the cases that local-params will be parsed to only contexts in which the
default parser is "lucene" or "func".
So if defType=edismax then q={!myparser ...} won't work. In that example, put the desired query parser into defType.
Another example is if deftype=edismax then hl.q={!myparser ...} won't work for the same reason. In that example,
either put the desired query parser into hl.qparser or set hl.qparser=lucene. Most users won't run into these cases
but some will and must change.
If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or something earlier.
(David Smiley)
- SOLR-11501: The edismax parser by default no longer allows subqueries that specify a Solr parser using either
local-params, or the older _query_ magic field trick. For example
{!prefix f=myfield v=enterp} or _query_:"{!prefix f=myfield v=enterp}" are not supported by default anymore.
If you want to allow power-users to do this, set uf=* _query_ or some other value that includes _query_.
If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or something earlier.
(David Smiley)
- New Features (33)
- SOLR-11448: Implement an option in collection commands to wait for final results.
(ab)
- SOLR-11072: Implement trigger for searchRate event type.
(ab)
- SOLR-11524: A new autoscaling/suggestions API end-point which gives autoscaling suggestions
(noble)
- SOLR-11519: Implement autoscaling suggestions for replica count violations
(noble)
- SOLR-11518: Implement autoscaling Suggestions for freedisk violations
(noble)
- SOLR-10132: A new optional facet.matches parameter to return facet buckets only
for terms that match a regular expression.
(Gus Heck, Christine Poerschke)
- SOLR-11520: Implement autoscaling suggestions for cores count violations
(noble)
- SOLR-11438: Solr should return rf when min_rf is specified for deletes as well as adds
(Erick Erickson)
- SOLR-11003: Support bi-directional syncing of cdcr clusters. We still only support active indexing in one cluster,
but have the ability to switch indexing clusters and cdcr will replicate correctly.
(Amrit Sarkar, Varun Thacker)
- SOLR-11538: Implement suggestions for port,ip_*, nodeRole,sysprop.*, metrics:*
(noble)
- SOLR-11487: Collection Aliases may now have metadata (currently an internal feature).
(Gus Heck, David Smiley)
- SOLR-11542: New TimeRoutedAliasUpdateProcessor URP that routes documents to another collection
in the same Alias defined set based on a time field (currently an internal feature).
(David Smiley)
- SOLR-9743: A new UTILIZENODE command
(noble)
- SOLR-11202: Implement a set-property command for AutoScaling API.
(ab, shalin)
- SOLR-11250: A new DefaultWrapperModel class for loading of large and/or externally stored
LTRScoringModel definitions.
(Yuki Yano, shalin, Christine Poerschke)
- SOLR-11662: New synonymQueryStyle option to configure whether SynonymQuery, a DisjunctionMaxQuery, or BooleanQuery
occurs over query terms that overlap their position.
(Doug Turnbull, David Smiley)
- SOLR-11429: Add loess Stream Evaluator to support local regression
(Joel Bernstein)
- SOLR-11568: Add matrix Stream Evaluator to support efficient matrix operations
(Joel Bernstein)
- SOLR-11566: Add transpose Stream Evaluator to support transposing of matrices
(Joel Bernstein)
- SOLR-11565: Add unit Stream Evaluator to support unitizing of vectors
(Joel Bernstein)
- SOLR-11567: Add triangularDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11593: Add support for covariance matrices to the cov Stream Evaluator
(Joel Bernstein)
- SOLR-10680: Add minMaxScale Stream Evaluator
(Joel Bernstein)
- SOLR-11599: Change normalize function to standardize and make it work with matrices
(Joel Bernstein)
- SOLR-11602: Add Markov Chain Stream Evaluator
(Joel Bernstein)
- SOLR-11607: Add grandSum, sumRows, sumColumns, scalarDivide, scalarMultiply, scalarAdd,
scalarSubtract Stream Evaluators
(Joel Bernstein)
- SOLR-11571: Add diff Stream Evaluator to support time series differencing
(Mathew Skaria, Joel Bernstein)
- SOLR-11570: Add support for correlation matrices to the corr Stream Evaluator
(Joel Bernstein)
- SOLR-11569: Add support for distance matrices to the distance Stream Evaluator
(Joel Bernstein)
- SOLR-11674: Support ranges in the probability Stream Evaluator
(Joel Bernstein)
- SOLR-11680: Add normalizeSum Stream Evaluator
(Joel Bernstein)
- SOLR-11697: Add geometricDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11485: Add olsRegress, spline and derivative Stream Evaluators
(Joel Bernstein)
- Bug Fixes (28)
- SOLR-11423: Overseer queue needs a hard cap (maximum size) that clients respect
(Scott Blum, Joshua Humphries, Noble Paul)
- SOLR-11445: Overseer should not hang when process bad message.
(Cao Manh Dat, shalin)
- SOLR-11447: ZkStateWriter should process commands in atomic.
(Cao Manh Dat, shalin)
- SOLR-11491: Support retrieval of cluster properties in HttpClusterStateProvider.
(ab)
- SOLR-11146: Small fixes to the mapping functions in the Analytics Component
(Houston Putman)
- SOLR-11326: A bootstrap of a target cluster does not need to download the tlog files from the source cluster
(Amrit Sarkar, Varun Thacker)
- SOLR-10874: Fix AIOOBE caused by FloatPayloadValueSource calling PostingsEnum.nextPosition()
more than the given term's frequency in overridden FloatDocValues.floatVal().
(Michael Kosten, Erik Hatcher, Steve Rowe)
- SOLR-11484: CloudSolrClient does not invalidate cache or retry for RouteException
(noble, hossman)
- SOLR-11532: Solr hits NPE when fl only contains DV fields and any of them is a spatial field.
(Cao Manh Dat, David Smiley)
- SOLR-11557: Fix SolrZkClient.checkInterrupted
(Tomás Fernández Löbbe)
- SOLR-11413: SolrGraphiteReporter fails to report metrics due to non-thread safe code.
(Erik Persson, ab)
- SOLR-9440: The ZkStateReader.removeCollectionStateWatcher method can cache a DocCollection reference and
never update it causing stale state to be returned in ClusterState.
(shalin)
- SOLR-11560: Specifying the replicationFactor parameter while restoring a collection would lead to extra tlog
and pull replicas being created
(Peter Szantai-Kis, Varun Thacker)
- SOLR-11586: Restored collection should use stateFormat=2 instead of 1.
(Varun Thacker)
- SOLR-11503: Collections created with legacyCloud=true cannot be opened if legacyCloud=false
(Erick Erickson)
- SOLR-11619: V2 requests that needed to be forwarded to other nodes would get an NPE.
(David Smiley)
- SOLR-11231: Guard against unset fields when performing language detection.
(Chris Beer via Steve Rowe)
- SOLR-11553: JSON Facet API: facet refinement could use UIF in phase one and DV uninversion in phase 2, resulting
in too much memory use.
(yonik)
- SOLR-11645: When the java commandline had exact duplicate arguments, the
admin UI dashboard would not display any commandline arguments.
(Webster Homer via Shawn Heisey)
- SOLR-11608: Correctly parse the new core-name in the V2 core rename API.
(Jason Gerlowski via Anshum Gupta)
- SOLR-11256: The queue size for ConcurrentUpdateSolrClient should default to 10 instead of throwing an
IllegalArgumentException.
(Jason Gerlowski, Anshum Gupta)
- SOLR-11616: Snapshot the segments more robustly such that segments created during a backup does does not fail the
operation
(Varun Thacker)
- SOLR-11687: SolrCore.getNewIndexDir falsely returns {dataDir}/index on any IOException reading index.properties
(Nikolay Martynov, Erick Erickson)
- SOLR-9137: bin/solr script ignored custom STOP_PORT on shutdown.
(Joachim Kohlhammer, Steve Rowe, Christine Poerschke)
- SOLR-11458: Improve error handling in MoveReplicaCmd to avoid potential loss of data.
(ab)
- SOLR-11590: Synchronize ZK connect/disconnect handling so that they are processed in linear order
(noble, Varun Thacker)
- SOLR-11664: JSON Facet API: range facets containing unique, hll, min, max aggregations over string fields
produced incorrect results since 7.0
(Volodymyr Rudniev, yonik)
- SOLR-11691: V2 requests for create-alias didn't work when the collections param was an array.
(Jason Gerlowski, Gus Heck, David Smiley, noble)
- Optimizations (5)
- SOLR-11285: Refactor autoscaling framework to avoid direct references to Zookeeper and Solr
to make testing and large scale simulations easier.
(ab)
- SOLR-11443: Remove the usage of workqueue for Overseer.
(Cao Manh Dat, Scott Blum)
- SOLR-11320: Lock autoscaling triggers when changes they requested are being made. This helps to
ensure that cluster is in a stable state before processing any new trigger events.
(ab)
- SOLR-11641: Change `frange` to default to `cost=100` so default behavior is to PostFilter if user specifies
`cache=false`
(hossman)
- SOLR-11595: SolrIndexSearcher.localCollectionStatistics is now faster by leveraging an existing
cached MultiFields. Noticeable when many fields are searched.
(David Smiley)
- Other Changes (19)
- SOLR-11478: Solr should remove itself from live_nodes in zk immediately on shutdown.
(Cao Manh Dat)
- SOLR-11145: Adds comprehensive unit tests for the Analytics Component v2
(Houston Putman)
- SOLR-11389: For Solr(Shard|Cluster)Reporter instances the SolrMetricManager.registerReporter
method is now called after the SolrCore or CoreContainer has been set for the instance.
(Christine Poerschke)
- SOLR-11464, SOLR-11493, SOLR-11511: Minor refactorings to DistributedUpdateProcessor.
(Gus Heck, David Smiley)
- SOLR-11444: Improved consistency of collection alias handling, and collection references that are
comma delimited lists of collections, across the various places collections can be referred to.
Updates are delivered to the first in a list. It's now possible to refer to a comma delimited list
of them in the path, ex: /solr/colA,colB/select?...
(David Smiley)
- SOLR-11165: Ref guide documentation for Autoscaling APIs, triggers, listeners, actions for Solr 7.1
(ab, Cassandra Targett, Noble Paul, shalin)
- SOLR-11562: Restore Solr logo ASCII-art in startup log by removing unnecessary default confdir logging
(janhoy)
- SOLR-11380: SolrJ must stream docs to server instead of writing to a buffer first
(noble)
- SOLR-11603: Remove unused (public) LTRScoringModel.hasParams() method.
(Christine Poerschke)
- SOLR-11606: Disable tests automatically if Mockito does not work with Java runtime (Java 10).
(Uwe Schindler)
- SOLR-11618: Tone down verbosity of BackupManager logging
(Varun Thacker)
- SOLR-11621: Fix spurious failures of TriggerIntegrationTest due to timing issues.
(shalin)
- SOLR-11628: Add documentation of maxRamMB for filter cache and query result cache.
(shalin)
- SOLR-11610: Refactored payload handling to use lucene's PayloadDecoder
framework
(Alan Woodward)
- SOLR-9120: Reduce log level for inconsequential NoSuchFileException that LukeRequestHandler may encounter
(hossman)
- SOLR-10469: Move CloudSolrClient.setParallelUpdates to its Builder.
(David Smiley)
- SOLR-11507: SOLR-11638: Randomize SolrTestCaseJ4.CloudSolrClientBuilder more, and simplify it.
(Jason Gerlowski, David Smiley)
- SOLR-11291: Factor out abstract metrics/SolrCore[Container]Reporter classes.
(Omar Abdelnabi, Christine Poerschke)
- SOLR-11713: Fixed CdcrUpdateLogTest.testSubReader() failure which was a test bug
(Varun Thacker, Amrit Sarkar)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.16
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.20.v20170531
Detailed Change List
- Upgrade Notes (7)
- 'autoAddReplicas' feature is ported to autoscaling framework. Existing users of this feature should not have
to change anything. Note these changes:
1. Behaviour: Changing the autoAddReplicas property from disabled to enabled using MODIFYCOLLECTION API
no longer replaces down replicas for the collection immediately. Instead, replicas are only added
if a node containing them went down while autoAddReplicas was enabled. The params
autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay are no longer used.
2. Deprecations: Enabling/disabling autoAddReplicas cluster wide with the API will be deprecated; use
suspend/resume trigger APIs with name='.auto_add_replicas' instead.
- SOLR-11195: shard and cluster metric reporter configuration now requires a class attribute.
If a reporter configures the group="shard" attribute then please also configure the
class="org.apache.solr.metrics.reporters.solr.SolrShardReporter" attribute.
If a reporter configures the group="cluster" attribute then please also configure the
class="org.apache.solr.metrics.reporters.solr.SolrClusterReporter" attribute.
- SOLR-11254: the abstract DocTransformer class now has an abstract score-less transform method variant.
- SOLR-11283: all Stream Evaluators in solrj.io.eval have been refactored to have a simplier and more
robust structure. This simplifies and condenses the code required to implement a new Evaluator and
makes it much easier for evaluators to handle differing data types (primitives, objects, arrays,
lists, and so forth).
(Dennis Gove)
- SOLR-10962: in the ReplicationHandler the master.commitReserveDuration sub-element is deprecated. Instead
please configure a direct commitReserveDuration element for use in all modes
(master, slave, cloud).
- SOLR-11482: RunExecutableListener was removed for security reasons. If you want to listen to
events caused by updates, commits, or optimize, write your own listener as native Java class
as part of a Solr plugin.
- SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... })
the resolving of external entities is now disallowed by default.
- New Features (39)
- SOLR-10339: New set-trigger and remove-trigger APIs for autoscaling.
(shalin)
- SOLR-10340: New set-listener and remove-listener API for autoscaling.
(shalin)
- SOLR-10358: New suspend-trigger and resume-trigger APIs for autoscaling.
(shalin)
- SOLR-10376: Implement autoscaling trigger for nodeAdded event.
(shalin)
- SOLR-10396: Implement trigger support for nodeLost event type
(Cao Manh Dat, shalin)
- SOLR-10496: New ComputePlanAction for autoscaling which uses the policy framework to compute cluster
operations upon a trigger fire.
(Noble Paul, shalin)
- SOLR-10965: New ExecutePlanAction for autoscaling which executes the operations computed by ComputePlanAction
against the cluster.
(shalin)
- SOLR-11019: Add addAll Stream Evaluator
(Joel Bernstein)
- SOLR-10996: Implement TriggerListener API
(ab, shalin)
- SOLR-11046: Add residuals Stream Evaluator
(Joel Bernstein)
- SOLR-10858: Make UUIDUpdateProcessorFactory as Runtime URP
(Amit Sarkar, noble)
- SOLR-11031: Implement SystemLogListener for autoscaling
(ab)
- SOLR-11205: Any metrics value can be directly accessed in autoscaling policies
(noble)
- SOLR-11199: Payloads supports an "operator" param. Supported operators are 'or', "phrase" ( default ).
A new "sum" function is also added. Example :
{!payload_score f=payload_field func=sum operator=or}A B C"
(Varun Thacker)
- SOLR-11215: Make a metric accessible through a single param.
(ab)
- SOLR-11244: Query DSL for Solr
(Cao Manh Dat)
- SOLR-11317: JSON Facet API: min/max aggregations on numeric fields are now typed better so int/long
fields return an appropriate integral type rather than a double.
(yonik)
- SOLR-11316: JSON Facet API: min/max aggregations are now supported on single-valued date fields.
(yonik)
- SOLR-10962: Make ReplicationHandler's commitReserveDuration configurable in SolrCloud mode.
(Ramsey Haddad, Christine Poerschke, hossman)
- SOLR-11382: Lucene's Geo3D (surface of sphere & ellipsoid) is now supported on spatial RPT fields by
setting spatialContextFactory="Geo3D". Furthermore, this is the first time Solr has out of the box
support for polygons.
(David Smiley)
- SOLR-11160: Add normalDistribution, uniformDistribution, sample and kolmogorovSmirnov Stream Evaluators
(Joel Bernstein)
- SOLR-11225: Add cumulativeProbability Stream Evaluator
(Joel Bernstein)
- SOLR-11241: Add discrete counting and probability Stream Evaluators
(Joel Bernstein)
- SOLR-11321: Add ebeAdd, ebeSubtract, ebeDivide, ebeMultiply, dotProduct and cosineSimilarity Stream Evaluators
(Joel Bernstein)
- SOLR-11338: Add Kendall's Tau-b rank and Spearmans rank correlation Stream Evaluators
(Joel Bernstein)
- SOLR-11339: Add Canberra, Chebyshev, Earth Movers and Manhattan Distance Stream Evaluators
(Joel Bernstein)
- SOLR-11342: Add sumDifference and meanDifference Stream Evaluators
(Joel Bernstein)
- SOLR-11350: Add primes Stream Evaluator
(Joel Bernstein)
- SOLR-11354: Add factorial and movingMedian Stream Evaluators
(Joel Bernstein)
- SOLR-11377: Add expMovingAverage (exponential moving average) and binomialCoefficient Stream Evaluators
(Mathew Skaria, Joel Bernstein)
- SOLR-11388: Add monteCarlo Stream Evaluator to support Monte Carlo simulations
(Joel Bernstein)
- SOLR-11398: Add weibullDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11400: Add logNormalDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11401: Add zipFDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11414: Add gammaDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11415: Add betaDistribution Stream Evaluator
(Joel Bernstein)
- SOLR-11436: Add polyfit and polyfitDerivative Stream Evaluators
(Joel Bernstein)
- SOLR-11439: Add harmonicFit Stream Evaluator
(Joel Bernstein)
- SOLR-11076: New /autoscaling/history API to return past cluster events and actions.
(ab)
- Bug Fixes (32)
- SOLR-10602: Triggers should be able to restore state from old instances when taking over.
(shalin)
- SOLR-10714: OverseerTriggerThread does not start triggers on overseer start until autoscaling
config watcher is fired.
(shalin)
- SOLR-10738: TriggerAction is initialised even if the trigger is never scheduled.
(shalin)
- SOLR-10668: fix NPE at sort=childfield(..) .. on absent values
(Mikhail Khludnev)
- SOLR-8984: EnumField's error reporting to now indicate the field name in failure log
(Lanny Ripple,
Ann Addicks via Ishan Chattopadhyaya)
- SOLR-11012: Fix three (JavaBinCodec not being closed) Resource Leak warnings.
(Christine Poerschke)
- SOLR-11026: MoveReplicaSuggester must check if the target becomes more 'loaded' than the source
if an operation is performed
(noble)
- SOLR-10994: CREATE, CREATESHARD & ADDREPLICA to use policy framework & support replica types
(noble)
- SOLR-11011: Assign.buildCoreName can lead to error in creating a new core when legacyCloud=false
(Cao Manh Dat)
- SOLR-10944: Get expression fails to return EOF tuple
(Susheel Kumar, Joel Bernstein)
- SOLR-6086: Replica is active during autowarming resulting in queries being sent to a replica that
may not have a registered searcher. This causes spikes in response times when adding a replica
in busy clusters.
(Ludovic Boutros, Timothy Potter, shalin)
- SOLR-11190: GraphQuery also supports string fields which are indexed=false and docValues=true. Please refer to the
Javadocs for DocValuesTermsQuery for it's performance characteristics.
(Karthik Ramachandran, Varun Thacker)
- SOLR-11084: Issue with starting script with solr.home (-s) == solr
(Leil Ireson, Amrit Sarkar via Erick Erickson)
- SOLR-11255: Fix occasional ConcurrentModificationException when using SolrInfoMBeanHandler.
(ab)
- SOLR-11272: fix NPE when EmbeddedSolrServer handles /admin/* request and so one
(Stephen Allen via Mikhail Khludnev)
- SOLR-11289: fix comma handling in terms component.
(yonik)
- SOLR-11164, SOLR-11180, SOLR-11220: Fix NullPointerException and always-returns-zero
contrib/ltr OriginalScoreFeature issues in SolrCloud mode.
(Yuki Yano, Jonathan Gonzalez, Ryan Yacyshyn, Christine Poerschke)
- SOLR-11278: Stopping CDCR should cancel a running bootstrap operation.
(Amrit Sarkar, shalin)
- SOLR-11293: Potential data loss in TLOG replicas when masterVersion equals zero
(noble, Cao Manh Dat)
- SOLR-10101: TestLazyCores hangs
(Erick Erickson)
- SOLR-11332: Fix sorting on 'enum' fieldTypes that use sortMissingFirst or sortMissingLast
(hossman)
- SOLR-11348: Fix the DIH database example
(James Dyer)
- SOLR-11363: JSON Facet API: repeated values in a numeric points field with docValues enabled
were double counted.
(Hossman, yonik)
- SOLR-11085: Improve resiliency of autoscaling actions against overseer restarts and operation failures.
(shalin)
- SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice.
(Luiz Armesto, Shawn Heisey, Erick Erickson)
- SOLR-11399: The UnifiedHighlighter was ignoring the hl.fragsize parameter when hl.bs.type=SEPARATOR
(Marc Morissette via David Smiley)
- SOLR-11224: SolrStream.close can hit an NPE
(Erick Erickson)
- SOLR-11278: Fix a race condition in the CDCR bootstrap process which could lead to bootstraps cancelling itself
(Amrit Sarkar, shalin, Varun Thacker)
- SOLR-11425: SolrClientBuilder does not allow infinite timeout (value 0).
(Peter Szantai-Kis via Mark Miller)
- SOLR-11449: MoveReplicaCmd mistakenly called registerCollectionStateWatcher on failure.
(ab)
- SOLR-11477: Disallow resolving of external entities in the XML query parser (defType=xmlparser).
(Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke)
- SOLR-12444: Updating a cluster policy fails
(noble)
- Optimizations (9)
- SOLR-10985: Remove unnecessary toString() calls in solr-core's search package's debug logging.
(Michael Braun via Christine Poerschke)
- SOLR-11000: Changes made via AutoScalingHandler should be atomic.
(ab, shalin)
- SOLR-11124: MoveReplicaCmd should skip deleting old replica in case of its node is not live
(Cao Manh Dat)
- SOLR-10769: Allow nodeAdded / nodeLost events to report multiple nodes in one event.
(ab)
- SOLR-11242: QueryParser: re-use the LookaheadSuccess exception.
(Michael Braun via David Smiley)
- SOLR-11314: FastCharStream: re-use the READ_PAST_EOF exception.
(Michael Braun via David Smiley)
- SOLR-8344: Decide default when requested fields are both column and row stored.
(Cao Manh Dat, David Smiley)
- SOLR-10285: Skip LEADER messages when there are leader only shards
(Cao Manh Dat, Joshua Humphries)
- SOLR-11424: When legacyCloud=false, cores should not publish itself as DOWN on startup.
(Cao Manh Dat)
- Other Changes (36)
- SOLR-10643: Throttling strategy for triggers and policy executions.
(shalin)
- SOLR-10764: AutoScalingHandler should validate policy and preferences before updating zookeeper.
(shalin)
- SOLR-10827: Factor out abstract FilteringSolrMetricReporter class.
(Christine Poerschke)
- SOLR-10957: Changed SolrCoreParser.init to use the resource loader from getSchema()
instead of the resource loader from getCore().
(Christine Poerschke)
- SOLR-10748: Make stream.body configurable and disabled by default
(janhoy)
- SOLR-10964: Reduce SolrIndexSearcher casting in LTRRescorer.
(Christine Poerschke)
- SOLR-11075: Refactor handling of params in CloudSolrStream and FacetStream
(Erick Erickson)
- SOLR-11052: Remove unnecessary Long-to-Integer and back casts in ReplicationHandler.
(Ramsey Haddad via Christine Poerschke)
- SOLR-11106: TestLBHttpSolrClient.testReliablity takes 30 seconds because of the wrong server name
(Kensho Hirasawa via Erick Erickson)
- SOLR-10338: Configure SecureRandom non blocking for tests.
(Mihaly Toth, hossman, Ishan Chattopadhyaya, via Mark Miller)
- SOLR-10916: Convert tests that extend LuceneTestCase and use MiniSolrCloudCluster
to instead extend SolrCloudTestCase.
(Steve Rowe)
- SOLR-11131: Document 'assert' as a command option in bin/solr, and bin/solr.cmd scripts.
(Jason Gerlowski via Anshum Gupta)
- SOLR-11140: Remove unused parameter in (private) SolrMetricManager.prepareCloudPlugins method.
(Omar Abdelnabi via Christine Poerschke)
- SOLR-11187: contrib/ltr TestModelManagerPersistence improvements.
(Yuki Yano via Christine Poerschke)
- SOLR-10397: Port 'autoAddReplicas' feature to the autoscaling framework and make it work with non-shared filesystems
(Cao Manh Dat, shalin)
- SOLR-11157: remove-policy must fail if a policy to be deleted is used by a collection
(noble)
- SOLR-11090: Add Replica.getProperty accessor.
(Christine Poerschke)
- SOLR-11061: Add a spins metric for data directory paths.
(ab)
- SOLR-11071: Improve TestIntervalFacets.testRandom
(Tomás Fernández Löbbe)
- SOLR-11195: Require class attribute for shard and cluster metric reporter configuration.
(Christine Poerschke)
- SOLR:10822: Share a Policy.session object between multiple collection admin calls to get the correct computations
(noble)
- SOLR-11249: Upgrade Jetty from 9.3.14.v20161028 to 9.3.20.v20170531
(Michael Braun via David Smiley)
- SOLR-11270: Automatically added SystemLogListener should not prevent deleting a trigger.
(ab)
- SOLR-5129: Timeout property for waiting ZK get started.
(Cao Manh Dat, Hrishikesh Gadre, Varun Thacker)
- SOLR-10628: Less verbose output from bin/solr commands.
(Jason Gerlowski, janhoy)
- SOLR-11240: Raise UnInvertedField internal limit.
(Toke Eskildsen)
- SOLR-11254: Add score-less (abstract) DocTransformer.transform method.
(Christine Poerschke)
- SOLR-11209: Upgrade HttpClient to 4.5.3.
(Hrishikesh Gadre via Mark Miller)
- SOLR-11322: JSON Facet API: instead of returning NaN, min & max aggregations omit
the value for any bucket with no values in the numeric field.
(yonik)
- SOLR-10990: Breakup QueryComponent.process method for readability.
(Christine Poerschke)
- SOLR-11132: Refactor common getSortField logic in various FieldTypes
(Jason Gerlowski, hossman)
- SOLR-11351: Make LTRScoringModel model more extensible.
(Christine Poerschke)
- SOLR-10451: Remove contrib/ltr/lib from lib includes in the techproducts example config
(sungjunyoung via janhoy)
- SOLR-11306: Fix inaccurate comments on docValues and StrField in the example schemas
(Tom Burton-West, Jason Gerlowski, Varun Thacker)
- SOLR-11482: RunExecutableListener was removed for security reasons.
(Michael Stepankin,
Olga Barinova, Uwe Schindler, Tomás Fernández Löbbe)
- SOLR-10335: Upgrade to Tika 1.16.
(Tim Allison, Tomás Fernández Löbbe via shalin)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.14.v20161028
- Bug Fixes (2)
- SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice.
(Luiz Armesto, Shawn Heisey, Erick Erickson)
- SOLR-11406: Solr 7.0 cannot read indexes from 6.x versions.
(Shawn Heisey, Steve Rowe)
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.14.v20161028
- Upgrading from Solr 6.x (40)
- All Trie* numeric and date field types have been deprecated in favor of *Point field types.
Point field types are better at range queries (speed, memory, disk), however simple field:value queries underperform
relative to Trie. Either accept this, or continue to use Trie fields.
This shortcoming may be addressed in a future release.
- The default response type is now JSON ("wt=json") instead of XML, and line indentation is now on by default
("indent=on"). If you expect the responses to your queries to be returned in the previous format (XML
format, no indentation), you must now you must now explicitly pass in "wt=xml" and "indent=off" as query
parameters, or configure them as defaults on your request handlers. See SOLR-10494 for more details.
- the cluster property 'legacyCloud' is set to false from 7.0. This means 'zookeeper is the truth' by
default. If an entry for a replica does not exist in the state.json, that replica cannot get
registered. This may affect users who use that feature where they bring up replicas and they are
automatically registered as a part of a shard. However, it is possible to fall back to the old behavior by
setting the property legacyCloud=true , in the cluster properties using the following command
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd clusterprop -name legacyCloud -val true
- HttpClientInterceptorPlugin is now HttpClientBuilderPlugin and must work with a
SolrHttpClientBuilder rather than an HttpClientConfigurer.
- HttpClientUtil now allows configuring HttpClient instances via SolrHttpClientBuilder
rather than an HttpClientConfigurer. Use of env variable SOLR_AUTHENTICATION_CLIENT_CONFIGURER
no longer works, please use SOLR_AUTHENTICATION_CLIENT_BUILDER
- SolrClient implementations now use their own internal configuration for socket timeouts,
connect timeouts, and allowing redirects rather than what is set as the default when
building the HttpClient instance. Use the appropriate setters on the SolrClient instance.
- HttpSolrClient#setAllowCompression has been removed and compression must be enabled as
a constructor param.
- HttpSolrClient#setDefaultMaxConnectionsPerHost and
HttpSolrClient#setMaxTotalConnections have been removed. These now default very
high and can only be changed via param when creating an HttpClient instance.
- Query time join with scoring {!join score=none} (even none) doesn't handle single value numeric fields.
Users are advised to convert these fields into string and reindex.
- Deprecated collection and configset methods on MiniSolrCloudCluster have been
removed
- Index-time boosts are not supported anymore. If any boosts are provided, they
will be ignored by the indexing chain. As a replacement, index-time scoring
factors should be indexed in a separate field and combined with the query
score using a function query.
- Deprecated method getNumericType() has been removed from FieldType. Use getNumberType() instead
- MBean names and attributes now follow hierarchical names used in metrics. This is reflected also in
/admin/mbeans and /admin/plugins output, and can be observed in the UI Plugins tab, because now all these
APIs get their data from the metrics API. The old (mostly flat) JMX view has been removed.
- <jmx> element in solrconfig.xml is no longer supported. Equivalent functionality can be configured in
solr.xml using <metrics><reporter ...> element and SolrJmxReporter implementation. Limited back-compatibility
is offered by automatically adding a default instance of SolrJmxReporter if it's missing, AND when a local
MBean server is found (which can be activated either via ENABLE_REMOTE_JMX_OPTS in solr.in.sh or via system
properties, eg. -Dcom.sun.management.jmxremote). This default instance exports all Solr metrics from all
registries as hierarchical MBeans. This behavior can be also disabled by specifying a SolrJmxReporter
configuration with a boolean init arg "enabled" set to "false". For a more fine-grained control users
should explicitly specify at least one SolrJmxReporter configuration.
- The sow (split-on-whitespace) request param now defaults to false (true in previous versions).
This affects the edismax and standard/"lucene" query parsers: if the sow param is not specified,
query text will not be split on whitespace before analysis. See
https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/ .
- Setting <solrQueryParser defaultOperator="..."/> in schema is no longer allowed and will cause an exception.
Please use "q.op" parameter on the request instead. For more details, see SOLR-10584.
- Setting <defaultSearchField> in schema is no longer allowed and will cause an exception.
Please use "df" parameter on the request instead. For more details, see SOLR-10585.
- The PostingsSolrHighlighter is deprecated. Furthermore, it now internally works via a re-configuration
of the UnifiedSolrHighlighter.
- Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and SpatialTermQueryPrefixTreeFieldType.
Instead, switch to LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField.
- The 'lucenePlusSort' QParser has been deprecated, and is no longer implicitly defined. If you wish to continue using
this QParser untill Solr 8, you must explicitly register it in your solrconfig.xml:
'<queryParser name="lucenePlusSort" class="solr.OldLuceneQParserPlugin"/>'
- In solrconfig.xml the deprecated <mergePolicy> and <mergeFactor> and <maxMergeDocs> elements have
been removed in favor of the <mergePolicyFactory> element introduced by SOLR-8621 in Solr 5.5.0.
- Custom SolrMetricReporter classes must implement a new, factored out doInit() method and the validate() method
must pass for all reporters including reporters that are not enabled, please see SOLR-10671 for details.
- TemplateUpdateRequestProcessorFactory uses {} instead of ${} for template
- SOLR-9565: The name of TemplateUpdateRequestProcessorFactory' is changed to 'template' from 'Template' and the
name of 'AtomicUpdateProcessorFactory' is changed to 'atomic' from 'Atomic'
- The default for eDismax parameter 'lowercaseOperators' now depends on the luceneMatchVersion setting in solrconfig.
It remains 'true' for luceneMatchVersion < 7.0.0 and changes to 'false' from 7.0.0. See also SOLR-4646
- The unused 'valType' option has been removed from ExternalFileField, if you have this in your schema you
can safely remove it. see SOLR-10929 for more details.
- Config sets basic_configs and data_driven_schema_configs have now been merged into _default. It has data driven nature
enabled by default, and can be turned off (after creating a collection) with:
curl http://host:8983/solr/mycollection/config -d '{"set-user-property": {"update.autoCreateFields":"false"}}'
Please see SOLR-10574 for details.
- The data driven config (now _default) for auto-creating fields earlier defaulted to "string" for text input.
Now the default is to use "text_general" and to add a copyField to the schema, copying to a "*_str" dynamic field,
with a cutoff at 256 characters. This enables full text search as well as faceting. See SOLR-9526 for more.
- SOLR-10123: The Analytics Component has been upgraded to support distributed collections, expressions over multivalued
fields, a new JSON request language, and more. DocValues are now required for any field used in the analytics expression
whereas previously docValues was not required. Please see SOLR-10123 for details.
- solrconfig.xml: <requestDispatcher handleSelect="..."> now defaults to false when luceneMatchVersion >= 7.0,
thus ignoring "qt". Regardless, "qt" is still used as a SolrJ special parameter that specifies the request handler
(tail URL path) to use. If you have request handlers without a leading '/', you can set handleSelect="true"
or better yet consider migrating.
- StandardRequestHandler is deprecated. Simply use SearchHandler instead.
- The parameter names 'fromNode' for MOVEREPLICA and 'source', 'target' for REPLACENODE have been deprecated and
replaced with 'sourceNode' and 'targetNode' instead. The old names will continue to work for back-compatibility
but they will be removed in 8.0. See SOLR-11068 for more details.
- All deperated methods of ClusterState (except getZkClusterStateVersion())
have been removed. Use DocCollection methods instead.
- SOLR-11023: EnumField has been deprecated in favor of new EnumFieldType.
- SOLR-11239: The use of maxShardsPerNode is not supported when a cluster policy is in effect or
when a collection specific policy is specified during collection creation.
- V2 APIs are now available at /api, in addition to /v2 (which is now deprecated). Legacy APIs continue to remain
available at /solr.
- Solr was tested and is compatible with the final release candidate of Java 9. All startup scripts
detect Java 9 correctly and setup Garbage Collector logging. If the configuration file contains
logging options that are no longer supported with Java 9, startup will fail.
- SOLR-10307: If starting Jetty without the Solr start script, you must now pass keystore and truststore
passwords via the env variables SOLR_SSL_KEY_STORE_PASSWORD and SOLR_SSL_TRUST_STORE_PASSWORD rather
than system properties.
- SOLR-10379: ManagedSynonymFilterFactory has been deprecated in favor of ManagedSynonymGraphFilterFactory.
- SOLR-10503: CurrencyField has been deprecated in favor of new CurrencyFieldType.
- New Features (51)
- SOLR-9857, SOLR-9858: Collect aggregated metrics from nodes and shard leaders in overseer.
(ab)
- SOLR-9835: Create another replication mode for SolrCloud
- SOLR-10356: Adds basic math Streaming Evaluators
(Dennis Gove)
- SOLR-10393: Adds UUID Streaming Evaluator
(Dennis Gove)
- SOLR-10046: Add UninvertDocValuesMergePolicyFactory class.
(Keith Laban, Christine Poerschke)
- SOLR-10547: JSON Facet API: Implement support for single-valued string fields for min/max aggregations.
(yonik)
- SOLR-10262: Add support for configurable metrics implementations.
(ab)
- SOLR-10431: Make it possible to invoke v2 api calls using SolrJ
(Cao Manh Dat, Noble Paul, shalin)
- SOLR-10233: Add support for different replica types, that can handle updates differently:
-
NRT: Writes updates to transaction log and indexes locally. Replicas of type “NRT” support NRT
(soft commits) and RTG. Any NRT replica can become a leader. This was the only type supported
in SolrCloud until now and it’s the default type.
-
TLOG: Writes to transaction log, but not to index, uses replication to copy segment files from the
shard leader. Any TLOG replica can become leader (by first applying all local transaction log
elements). If a replica is of type TLOG but is also the leader, it will behave as a NRT. This
is exactly what was added in SOLR-9835 (non-realtime replicas), just the API and naming changes.
-
PULL: Doesn’t index or writes to transaction log, just replicates from the shard leader. PULL replicas
can’t become shard leaders (i.e., if there are only PULL replicas in the collection at some point,
updates will fail same as if there is no leaders, queries continue to work), so they don’t even
participate in elections.
(Tomás Fernández Löbbe)
- SOLR-10278: A new DSL to set cluster-wide preferences and policies on how to allocate replicas to nodes
(noble, shalin)
- SOLR-10373: Implement read API for autoscaling configuration at /admin/autoscaling or
/cluster/autoscaling paths.
(shalin)
- SOLR-10677: Expose a diagnostics API to return nodes sorted by load in descending order and
any policy violations.
(shalin)
- SOLR-7452: Refinement for JSON Facet API: Adding refine:true to any terms facet will cause an
additional distributed search phase (overlapped with field retrieval) that requests additional info
for top facet buckets from shards that did not previously contribute to that bucket.
This will correct counts (and other statistics) for those top buckets collected in the first
phase.
(yonik)
- SOLR-10433: CollectionAdmin requests in SolrJ to support V2 calls
(noble)
- SOLR-9989: Add support for PointFields in FacetModule (JSON Facets)
(Cao Manh Dat)
- SOLR-10406: v2 API error messages list the URL request path as /solr/____v2/... when the original path was /v2/...
(Cao Manh Dat, noble)
- SOLR-10574: New _default config set replacing basic_configs and data_driven_schema_configs.
(Ishan Chattopadhyaya, noble, shalin, hossman, David Smiley, janhoy, Alexandre Rafalovitch)
- SOLR-10272: Use _default config set if no collection.configName is specified with CREATE
(Ishan Chattopadhyaya)
- SOLR-9526: Data driven schema now indexes text field "foo" as both "foo" (text_general) and as "foo_str" (string)
to facilitate both search and faceting. AddSchemaFieldsUpdateProcessor now has the ability to add a "copyField" to
the type mappings, with an optional maxChars limitation. You can also define one typeMappings as default.
This also solves issues SOLR-8495, SOLR-6966, and SOLR-7058
(janhoy, Steve Rowe, hossman, Alexandre Rafalovitch, Shawn Heisey, Cao Manh Dat)
- SOLR-10123: Upgraded the Analytics Component to version 2.0 which now supports distributed collections, expressions over
multivalued fields, a new JSON request language, and more. DocValues are now required for any field used in the analytics
expression whereas previously docValues was not required. Please see SOLR-10123 for details.
(Houston Putman)
- SOLR-10282: bin/solr support for enabling Kerberos authentication
(Ishan Chattopadhyaya, Jason Gerlowski)
- SOLR-11093: Add support for PointFields for {!graph} query.
(yonik)
- SOLR-10845: Add support for PointFields to {!graphTerms} query that is internally
used by some graph traversal streaming expressions.
(yonik)
- SOLR-10939: Add support for PointsFields to {!join} query. Joined fields should
also have docValues enabled.
(yonik)
- SOLR-11173 TermsComponent support for Points fields.
(yonik)
- SOLR-10849: MoreLikeThisComponent should expose setMaxDocFreqPct (maxDoc
frequency percentage).
(Dawid Weiss)
- SOLR-10307: Allow Passing SSL passwords through environment variables.
(Mano Kovacs, Michael Suzuki via Mark Miller)
- SOLR-10379: Add ManagedSynonymGraphFilterFactory, deprecate ManagedSynonymFilterFactory.
(Steve Rowe)
- SOLR-10479: Adds support for HttpShardHandlerFactory.loadBalancerRequests(MinimumAbsolute|MaximumFraction)
configuration.
(Ramsey Haddad, Daniel Collins, Christine Poerschke)
- SOLR-3702: concat(...) function query
(Andrey Kudryavtsev via Mikhail Khludnev)
- SOLR-10767: Add movingAvg Stream Evaluator
(Joel Bernstein)
- SOLR-10813: Add arraySort Stream Evaluator
(Joel Bernstein)
- SOLR-10696: Add cumulative probability function
(Joel Bernstein)
- SOLR-10765: Add anova Stream Evaluator
(Joel Bernstein)
- SOLR-10754: Add hist Stream Evaluator
(Joel Bernstein)
- SOLR-10753: Add array Stream Evaluator
(Joel Bernstein)
- SOLR-10747: Allow /stream handler to execute Stream Evaluators directly
(Joel Bernstein)
- SOLR-10743: Add sequence StreamEvaluator
(Joel Bernstein)
- SOLR-10684: Add finddelay Stream Evaluator
(Joel Bernstein)
- SOLR-10731: Add knn Streaming Expression
(Joel Bernstein)
- SOLR-10724: Add describe Stream Evaluator
(Joel Bernstein)
- SOLR-10693: Add copyOfRange Stream Evaluator
(Joel Bernstein)
- SOLR-10623: Add sql Streaming Expression
(Joel Bernstein)
- SOLR-10661: Add copyOf Stream Evaluator
(Joel Bernstein)
- SOLR-10663: Add distance Stream Evaluator
(Joel Bernstein)
- SOLR-10664: Add scale Stream Evaluator
(Joel Bernstein)
- SOLR-10666: Add rank transformation Stream Evaluator
(Joel Bernstein)
- SOLR-10662: Add length Stream Evaluator
(Joel Bernstein)
- SOLR-10660: Add reverse Stream Evaluator
(Joel Bernstein)
- SOLR-9910: Add solr/solr.cmd parameter to append jetty parameters to the start script.
(Mano Kovacs via Mark Miller)
- SOLR-6671: Possible to set solr.data.home property as root dir for all data
(janhoy, Shawn Heisey, Mark Miller)
- Bug Fixes (55)
- SOLR-9262: Connection and read timeouts are being ignored by UpdateShardHandler after SOLR-4509.
(Mark Miller, shalin)
- SOLR-9837: Fix 55% performance regression of FieldCache uninvert time of
numeric fields.
(yonik)
- SOLR-10408: v2 API introspect should return useful message for non-existent command
(Cao Manh Dat)
- SOLR-10411: v2 Collection API "modify" command specification's replicationFactor property is incorrectly typed as string,
should be integer
(Cao Manh Dat)
- SOLR-10405: v2 API introspect response contains multiple copies of the experimental format WARNING
(Cao Manh Dat)
- SOLR-10412: v2 API: many API command specification properties are typed "number" but should instead be typed "integer"
(Steve Rowe, Cao Manh Dat)
- SOLR-10413: v2 API: parsed JSON type should be coerced to expected type
(Cao Manh Dat, Noble Paul)
- SOLR-10223: Allow running examples as root on Linux with -force option
(janhoy)
- SOLR-10830: Solr now correctly enforces that the '_root_' field has the same fieldType as the
uniqueKey field. With out this enforcement, child document updating was unreliable.
(hossman)
- SOLR-10876: Regression in loading runtime UpdateRequestProcessors like TemplateUpdateProcessorFactory
(noble)
- SOLR-10886: Using V2Request.process(solrClient) method throws NPE if the API returns an error
(Cao Manh Dat)
- SOLR-10921: Work around the static Lucene BooleanQuery.maxClauseCount that causes Solr's maxBooleanClauses
setting behavior to be "last core wins". This patch sets BooleanQuery.maxClauseCount to its maximum value,
thus disabling the global check, and replaces it with specific checks where desired via
QueryUtils.build().
(yonik)
- SOLR-10948: Fix extraction component to treat DatePointField the same as TrieDateField
(hossman)
- SOLR-10506: Fix memory leak (upon collection reload or ZooKeeper session expiry) in ZkIndexSchemaReader.
(Torsten Bøgh Köster, Christine Poerschke, Jörg Rathlev, Mike Drob)
- SOLR-6807: CloudSolrClient's ZK state version check with the server was ignored when handleSelect=false
(David Smiley)
- SOLR-10878: MOVEREPLICA command may lose data when replicationFactor is 1.
(ab, shalin)
- SOLR-10879: DELETEREPLICA and DELETENODE commands should prevent data loss when
replicationFactor is 1.
(ab)
- SOLR-10983: Fix DOWNNODE -> queue-work explosion
(Scott Blum, Joshua Humphries)
- SOLR-10826: Fix CloudSolrClient to expand the collection parameter correctly
(Tim Owen via Varun Thacker)
- SOLR-11039: Next button in Solr admin UI for collection list pagination does not work.
(janhoy)
- SOLR-11041: MoveReplicaCmd do not specify ulog dir in case of HDFS
(Cao Manh Dat)
- SOLR-11045: The new replica created by MoveReplica will have to have same name and coreName as the
old one in case of HDFS
(Cao Manh Dat)
- SOLR-11043: Fix facet.range.method=dv and interval facets on single-valued float fields with negative values.
(Tomás Fernández Löbbe, Steve Rowe)
- SOLR-11073: Fix overflow in interval faceting when querying Long limits (e.g. (Long.MAX_VALUE TO Long.MAX_VALUE])
(Tomás Fernández Löbbe)
- SOLR-11057: Fix overflow in point range queries when querying the type limits
(e.g. q=field_i:{Integer.MAX_VALUE TO Integer.MAX_VALUE])
(Tomás Fernández Löbbe)
- SOLR-11136: Fix solrj XMLResponseParser when nested docs transformer is used with indented XML
(hossman)
- SOLR-11130: V2Request in SolrJ should return the correct collection name so that the request is forwarded to the
correct node
(noble)
- SOLR-11151: SolrInfoMBeanHandler.getDiff() ADD case non-functional: NPE when a bean value goes from null -> non-null.
(Steve Rowe)
- SOLR-11154: Child documents' return fields now include useDocValuesAsStored fields
(Mohammed Sheeri Shaketi Nauage via
Ishan Chattopadhyaya)
- SOLR-11163: Fix contrib/ltr Normalizer persistence after solr core reload or restart.
(Yuki Yano via Christine Poerschke)
- SOLR-11182: A split shard failure on IOException should be logged
(Varun Thacker)
- SOLR-10353: TestSQLHandler reproducible failure: No match found for function signature min(<NUMERIC>)
(Kevin Risden)
- SOLR-11221: SolrJmxReporter broken on core reload. This resulted in some or most metrics not being reported
via JMX after core reloads, depending on timing.
(ab)
- SOLR-11235: Some SolrCore metrics should check if core is closed before reporting.
(ab)
- SOLR-10698: StreamHandler should allow connections to be closed early
(Joel Bernstein, Varun Thacker, Erick Erickson)
- SOLR-11243: Replica Placement rules are ignored if a cluster policy exists.
(shalin)
- SOLR-11268: AtomicUpdateProcessor complains missing UpdateLog
(noble, Ishan Chattopadhyaya)
- SOLR-8689: Fix bin/solr.cmd so it can run properly on Java 9
(Uwe Schindler, hossman)
- SOLR-10723 JSON Facet API: resize() implemented incorrectly for CountSlotAcc, HllAgg.NumericAcc
resulting in exceptions when using a hashing faceting method and sorting by hll(numeric_field).
(yonik)
- SOLR-10719: Creating a core.properties fails if the parent of core.properties is a symlinked dierctory
(Erick Erickson)
- SOLR-10360: Solr HDFS snapshot export fails due to FileNotFoundException error when using MR1 instead of
yarn.
(Hrishikesh via Mark Miller)
- SOLR-10137: Ensure that ConfigSets created via API are mutable.
(Hrishikesh via Mark Miller)
- SOLR-10829: Fixed IndexSchema to enforce that uniqueKey can not be Points based for correctness
(hossman)
- SOLR-10836: The query parsers igain, significantTerms, and tlogit (used by streaming expressions by
the same name) might throw a NullPointerException if the referenced field had no indexed data in some
shards. The fix included an optimization to use Solr's cached AtomicReader instead of re-calculating.
(David Smiley)
- SOLR-10715: /v2/ should not be an alias for /v2/collections
(Cao Manh Dat)
- SOLR-10835: Add support for point fields in Export Handler
(Tomás Fernández Löbbe)
- SOLR-10704: REPLACENODE may cause data loss when replicationFactor is 1.
(ab, shalin)
- SOLR-10833: Point numeric fields should throw SolrException(BAD_REQUEST) for malformed numbers in queries.
Trie numeric fields should throw SolrException(BAD_REQUEST) for malformed docValues range queries.
(hossman, Tomás Fernández Löbbe)
- SOLR-10832: Fixed VersionInfo.getMaxVersionFromIndex when using PointsField with indexed="true"
(hossman)
- SOLR-10763: Admin UI replication tab sometimes empty when failed replications
(janhoy, Bojan Vitnik)
- SOLR-10824: fix NPE ExactSharedStatsCache, fixing maxdocs skew for terms which are absent at one of shards
- when using one of Exact*StatsCache
(Mikhail Khludnev)
- SOLR-10963: Fix example json in MultipleAdditiveTreesModel javadocs.
(Stefan Langenmaier via Christine Poerschke)
- SOLR-10914: RecoveryStrategy's sendPrepRecoveryCmd can get stuck for 5 minutes if leader is unloaded.
(shalin)
- SOLR-11198: downconfig downloads empty file as folder
(Erick Erickson)
- Optimizations (8)
- SOLR-4509: Move to non deprecated HttpClient impl classes to remove stale connection
check on every request and move connection lifecycle management towards the client.
(Ryan Zezeski, Mark Miller, Shawn Heisey, Steve Davids)
- SOLR-9255: Rename SOLR_AUTHENTICATION_CLIENT_CONFIGURER -> SOLR_AUTHENTICATION_CLIENT_BUILDER
(janhoy)
- SOLR-9579: Make Solr's SchemaField implement Lucene's IndexableFieldType, removing the
creation of a Lucene FieldType every time a field is indexed.
(John Call, yonik)
- SOLR-9822: JSON Facet API: Recover performance lost due to the DocValues transition to
an iterator API (LUCENE-7407). This only fixes calculating counts for single-valued
string fields from the FieldCache, resulting in up to 56% better throughput for those cases.
(yonik)
- SOLR-10727: Avoid polluting the filter cache for certain types of faceting (typically ranges) when
the base docset is empty.
(David Smiley)
- SOLR-11070: Make docValues range queries behave the same as Trie/Point fields for Double/Float Infinity cases
(Tomás Fernández Löbbe, Andrey Kudryavtsev)
- SOLR-10634: JSON Facet API: When a field/terms facet will retrieve all buckets (i.e. limit:-1)
and there are no nested facets, aggregations are computed in the first collection phase
so that the second phase which would normally involve calculating the domain for the bucket
can be skipped entirely, leading to large performance improvements.
(yonik)
- SOLR-10722: Speed up Solr's use of the UnifiedHighlighter be re-using FieldInfos.
(David Smiley)
- Other Changes (89)
- SOLR-10236: Removed FieldType.getNumericType(). Use getNumberType() instead.
(Tomás Fernández Löbbe)
- SOLR-10347: Removed index level boost support from "documents" section of the admin UI
(Amrit Sarkar via
Tomás Fernández Löbbe)
- SOLR-9959: SolrInfoMBean category and hierarchy cleanup. Per-component statistics are now obtained from
the metrics API, legacy JMX support has been replaced with SolrJmxReporter functionality. Several reporter
improvements (support for multiple prefix filters, "enabled" flag, reuse of service clients).
(ab)
- SOLR-10418: Expose safe system properties via metrics API as 'system.properties' in 'solr.jvm' group.
Add support for selecting specific properties from any compound metric using 'property' parameter to
/admin/metrics handler.
(ab)
- SOLR-10557: Make "compact" format default for /admin/metrics.
(ab)
- SOLR-10310: By default, stop splitting on whitespace prior to analysis
in edismax and standard/"lucene" query parsers.
(Steve Rowe)
- SOLR-10647: Move the V1 <-> V2 API mapping to SolrJ
(noble)
- SOLR-10584: We'll now always throw an exception if defaultOperator is found in schema. This config has
been deprecated since 3.6, and enforced for new configs since 6.6
(janhoy)
- SOLR-10585: We'll now always throw an exception if defaultSearchField is found in schema. This config has
been deprecated since 3.6, and enforced for new configs since 6.6
(janhoy, David Smiley)
- SOLR-10414: RecoveryStrategy is now a Runnable instead of a Thread.
(Tomás Fernández Löbbe)
- SOLR-10042: Delete old deprecated Admin UI, leaving the AngularJS UI the only one supported
(janhoy)
- SOLR-10378: Clicking Solr logo on AdminUI shows blank page
(Takumi Yoshida via janhoy)
- SOLR-10700: Deprecated and converted the PostingsSolrHighlighter to extend UnifiedSolrHighlighter and thus no
longer use the PostingsHighlighter. It should behave mostly the same.
(David Smiley)
- SOLR-10710: Fix LTR failing tests.
(Diego Ceccarelli via Tomás Fernández Löbbe)
- SOLR-10755: delete/refactor many solrj deprecations
(hossman)
- SOLR-10752: replicationFactor (nrtReplicas) default is 0 if tlogReplicas is specified when creating a collection
(Tomás Fernández Löbbe)
- SOLR-10757: delete/refactor/cleanup CollectionAdminRequest deprecations
(hossman)
- SOLR-10793: BlobHandler should have a well-known permission name
(noble)
- SOLR-10744: Update noggit to newer version (0.8)
(noble)
- SOLR-10792: Deprecate and remove implicit registration of "lucenePlusSort" aka OldLuceneQParser
(hossman)
- SOLR-10791: Remove deprecated options in SSLTestConfig
(hossman)
- LUCENE-7852: Correct copyright year(s) in solr/LICENSE.txt file.
(Christine Poerschke, Steve Rowe)
- SOLR-8668: In solrconfig.xml remove <mergePolicy> (and related <mergeFactor> and <maxMergeDocs>)
support in favor of the <mergePolicyFactory> element introduced by SOLR-8621 in Solr 5.5.0.
(Christine Poerschke, hossman)
- SOLR-10799: Extracted functionality to collect eligible replicas from HttpShardHandler.prepDistributed()
to a new method
(Domenico Fabio Marino via Tomás Fernández Löbbe)
- SOLR-10801: Remove several deprecated methods that were exposed to plugin writers
(hossman)
- SOLR-10671: Add abstract doInit method to the SolrMetricReporter base class.
(Christine Poerschke, Anshum Gupta)
- SOLR-10713: Ignore .pid and .out files in solr working directory
(Jason Gerlowski via Mike Drob)
- SOLR-10764: AutoScalingHandler should validate policy and preferences before updating zookeeper.
(shalin)
- SOLR-10782: Improve error handling and tests for Snitch and subclasses and general cleanups.
(Noble Paul, shalin)
- SOLR-10419: All collection APIs should use the new Policy framework for replica placement.
(Noble Paul, shalin)
- SOLR-10800: Factor out HttpShardHandler.transformReplicasToShardUrls from HttpShardHandler.prepDistributed.
(Domenico Fabio Marino, Christine Poerschke)
- SOLR-10779: JavaBinCodec should use close consistently rather than having marshal() and close() call finish()
(which closes the underlying stream).
(Erick Erickson)
- SOLR-9623: Disable remote streaming in example configs by default. Adjust Upload Limit defaults
(janhoy, yonik)
- SOLR-4646: eDismax lowercaseOperators now defaults to "false" for luceneMatchVersion >= 7.0.0
(janhoy, David Smiley)
- SOLR-8256: Set legacyCloud=false as default
(Cao Manh Dat)
- SOLR-10929: Removed unused 'valType' option from ExternalFileField
(hossman)
- SOLR-10864: Simplified how Trie vs Points based numerics are randomized by SolrTestCaseJ4 by adding a static
option to PointsField to ignore 'precisionStep' attribute. This change also begins to attempt to randomize
'docValues' on numeric field types unless tests explicity enable/disable them.
(hossman, Steve Rowe)
- LUCENE-7883: Solr no longer uses the context class loader when resolving
resources, they are only resolved against Solr's own or "core" class loader
by default.
(Uwe Schindler)
- SOLR-10915: Make builder based SolrClient constructors to be the only valid way to construct client objects and
increase the visibility of builder elements to be protected so extending the builder, and the clients is possible.
(Jason Gerlowski, Anshum Gupta)
- SOLR-10823: Add reporting period to SolrMetricReporter base class.
(Christine Poerschke)
- SOLR-10807: Randomize Points based numeric field types in all existing tests & test schemas
where Trie numerics were previously hardcoded.
(hossman, Steve Rowe, Anshum Gupta)
- SOLR-6807: Changed requestDispatcher's handleSelect to default to false, thus ignoring "qt".
Simplified configs to not refer to handleSelect or "qt". Switch all tests that assumed true to assume false
(added leading '/' in request handlers). Switch all tests referring to "standard" request handler to
instead refer to "/select" with SearchHandler. Deprecated the old StandardRequestHandler.
(David Smiley)
- SOLR-10456: Deprecate timeout related setters from SolrClients, and replace with Builder based implementation
(Jason Gerlowski, Anshum Gupta)
- SOLR-11004: Consolidate SolrClient builder code in an abstract base class
(Jason Gerlowski, Anshum Gupta)
- SOLR-10967: Cleanup the default configset. The changes involve some documentation corrections, removing the currency
field from the examples and a few dynamic fields
(Varun Thacker)
- SOLR-11016: Fix TestCloudJSONFacetJoinDomain test-only bug
(hossman)
- SOLR-11021: The elevate.xml config-file is made optional in the ElevationComponent.
The default configset doesn't ship with a elevate.xml file anymore
(Varun Thacker)
- SOLR-10898: Fix SOLR-10898 to not deterministicly fail 1/512 runs
(hossman)
- SOLR-10796: TestPointFields: increase randomized testing of non-trivial values.
(Steve Rowe)
- SOLR-11068: MOVEREPLICA and REPLACENODE API parameter names are now 'sourceNode' and 'targetNode'. The old names
viz. 'fromNode' for MOVEREPLICA and 'source', 'target' for REPLACENODE have been deprecated.
(shalin)
- SOLR-11088: Fix sporadic failures of MetricsHandlerTest.testPropertyFilter on jenkins
(shalin)
- SOLR-11037: Refactor to provide NodeConfig.getSolrDataHome internal API.
(ab, janhoy, shalin)
- SOLR-11119: Switch from Trie to Points field types in the .system collection schema.
(Steve Rowe)
- SOLR-10494: Make default response format JSON (wt=json), and also indent text responses formats
(indent=on) by default
(Trey Grainger & Cassandra Targett via hossman)
- SOLR-10760,SOLR-11315,SOLR-11313: Remove trie field types and fields from example schemas.
(Steve Rowe)
- SOLR-11056: Add random range query test that compares results across Trie*, *Point and DocValue-only fields
(Tomás Fernández Löbbe)
- SOLR-10926: Increase the odds of randomly choosing point fields in our SolrTestCaseJ4 numeric type randomization.
(hossman, Steve Rowe)
- SOLR-10846: ExternalFileField/FloatFieldSource should throw a clear exception on initialization with
a Points-based keyField, which is not supported.
(hossman, Steve Rowe)
- SOLR-11155: /analysis/field and /analysis/document requests should support points fields.
(Jason Gerlowski, Steve Rowe)
- SOLR-10756: Undeprecate ZkStateReader.updateClusterState(), mark as @lucene.internal, and rename to
forciblyRefreshAllClusterStateSlow().
(hossman, shalin, Steve Rowe)
- SOLR-11036: Separately report disk space metrics for solr.data.home and core root directory.
(ab)
- SOLR-10919: ord & rord functions give confusing errors with PointFields.
(hossman, Steve Rowe)
- SOLR-10847: Provide a clear exception when attempting to use the terms component with points fields.
(hossman, Steve Rowe)
- SOLR-9321: Remove deprecated methods of ClusterState.
(Jason Gerlowski, ishan, Cao Manh Dat)
- SOLR-10033: When attempting to facet with facet.mincount=0 over points fields, raise mincount to 1
and log a warning.
(Steve Rowe)
- SOLR-11178: Change error handling in AutoScalingHandler to be consistent w/ other APIs
(noble)
- SOLR-11023: Added EnumFieldType, a non-Trie-based version of EnumField, and deprecated EnumField
in favor of EnumFieldType.
(hossman, Steve Rowe)
- SOLR-10803: Mark all Trie/LegacyNumeric based fields @deprecated in Solr7.
(Steve Rowe)
- SOLR-10821: Ref guide documentation for Autoscaling
(Noble Paul, Cassandra Targett, shalin)
- SOLR-11239: A special value of -1 can be specified for 'maxShardsPerNode' to denote that
there is no limit. The bin/solr script send maxShardsPerNode=-1 when creating collections. The
use of maxShardsPerNode is not supported when a cluster policy is in effect or when a
collection specific policy is specified during collection creation.
(Noble Paul, shalin)
- SOLR-11183: V2 APIs are now available at /api endpoint.
(Ishan Chattopadhyaya)
- SOLR-10617: JDBCStream accepts columns of type TIME, DATE & TIMESTAMP as well as CLOBs and decimal
numeric types
(James Dyer)
- SOLR-10400: Replace (instanceof TrieFooField || instanceof FooPointField) constructs with
FieldType.getNumberType() or SchemaField.getSortField() where appropriate.
(hossman, Steve Rowe)
- SOLR-10438: Assign explicit useDocValuesAsStored values to all points field types in
schema-point.xml/TestPointFields.
(hossman, Steve Rowe)
- LUCENE-7705: Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length.
(Amrit Sarkar via Erick Erickson)
- SOLR-10659: Remove ResponseBuilder.getSortSpec use in SearchGroupShardResponseProcessor.
(Judith Silverman via Christine Poerschke)
- SOLR-10741: Factor out createSliceShardsStr method from HttpShardHandler.prepDistributed.
(Domenico Fabio Marino via Christine Poerschke)
- SOLR-10790: Fix warnings in Assign.java and ReplicaAssigner.java classes.
(Christine Poerschke)
- SOLR-8437: Improve RAMDirectory details in sample solrconfig files
(Mark Miller, Varun Thacker)
- SOLR-8762: return child docs in DIH debug
(Gopikannan Venugopalsamy via Mikhail Khludnev)
- SOLR-10501: Test sortMissing{First,Last} with points fields.
(Steve Rowe)
- SOLR-10761: Switch trie numeric/date fields to points in data-driven-enabled example and test schemas.
(Steve Rowe)
- SOLR-10851: SolrClients should clarify expectations for solrServerUrl parameter
(Jason Gerlowski
via Tomás Fernández Löbbe)
- SOLR-10891: BBoxField should support point-based number sub-fields.
(Steve Rowe)
- SOLR-10834: Fixed tests and test configs to stop using numeric uniqueKey fields
(hossman)
- SOLR-10883: Ref guide: Escape replacement substitutions, e.g. => to right arrow, so that they are
rendered visibly in the PDF. Also add .adoc file checks to the top-level validate target, including
for the invisible substitutions PDF problem.
(Steve Rowe)
- SOLR-10503,SOLR-10502: Deprecate CurrencyField in favor of new CurrencyFieldType, which works
with point fields and provides control over dynamic fields used for the raw amount and currency
code sub-fields.
(hossman, Steve Rowe)
- SOLR-11261, SOLR-10966: Upgrade to Hadoop 2.7.4 to fix incompatibility with Java 9.
(Uwe Schindler)
- SOLR-11324: Clean up mention of trie fields in documentation and source comments.
(Steve Rowe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.14.v20161028
- New Features (1)
- SOLR-12530: Ability to disable configset upload via -Dconfigset.upload.enabled=false startup parameter
(Ishan Chattopadhyaya)
- Bug Fixes (1)
- SOLR-12450: Don't allow referal to external resources in various config files (CVE-2018-8026).
(Yuyang Xiao, Uwe Schindler)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.17
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.11
- Jetty 9.4.10.v20180503
- Bug Fixes (1)
- SOLR-12316: Do not allow to use absolute URIs for including other files in solrconfig.xml and schema parsing.
(Ananthesh, Ishan Chattopadhyaya, Uwe Schindler)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.17
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.11
- Jetty 9.4.8.v20171121
- Bug Fixes (3)
- SOLR-11971: Don't allow referal to external resources in DataImportHandler's dataConfig request parameter.
(麦 香浓郁, Uwe Schindler)
- SOLR-11503: Collections created with legacyCloud=true cannot be opened if legacyCloud=false
(Erick Erickson)
- SOLR-11993: LeaderInitiatedRecoveryThread does not retry on UnknownHostException
(shalin, Cao Manh Dat, Steve Rowe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.16
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.14.v20161028
- Upgrade Notes (2)
- SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... })
the resolving of external entities is now disallowed by default.
- SOLR-11482: RunExecutableListener was deprecated and is disabled by default for
security reasons. Legacy applications still using it must explicitely pass
'-Dsolr.enableRunExecutableListener=true' to the Solr command line.
Be aware that you should really disable API-based config editing at the same
time, using '-Ddisable.configEdit=true'!
(Uwe Schindler)
- Bug Fixes (2)
- SOLR-11477: Disallow resolving of external entities in the XML query parser (defType=xmlparser).
(Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke)
- SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice.
(Luiz Armesto, Shawn Heisey, Erick Erickson)
- Other Changes (1)
- SOLR-10335: Upgrade to Tika 1.16.
(Tim Allison, Tomás Fernández Löbbe via shalin)
- Bug Fixes (11)
- SOLR-10857: standalone Solr loads UNLOADed core on request
(Erick Erickson, Mikhail Khludnev)
- SOLR-11024: ParallelStream should set the StreamContext when constructing SolrStreams
(Joel Bernstein)
- SOLR-10908: CloudSolrStream.toExpression incorrectly handles fq clauses
(Rohit via Erick Erickson)
- SOLR-11177: CoreContainer.load needs to send lazily loaded core descriptors to the proper list rather than send
them all to the transient lists.
(Erick Erickson)
- SOLR-11122: Creating a core should write a core.properties file first and clean up on failure
(Erick Erickson)
- SOLR-10910: Clean up a few details left over from pluggable transient core and untangling
CoreDescriptor/CoreContainer references
(Erick Erickson)
- SOLR-10721: Provide a way to know when Core Discovery is finished and when all async cores are done loading
(Erick Erickson)
- SOLR-11069: CDCR bootstrapping can get into an infinite loop when a core is reloaded
(Amrit Sarkar, Erick Erickson)
- SOLR-11221: SolrJmxReporter broken on core reload. This resulted in some or most metrics not being reported
via JMX after core reloads, depending on timing.
(ab)
- SOLR-11261, SOLR-10966: Upgrade to Hadoop 2.7.4 to fix incompatibility with Java 9.
(Uwe Schindler)
- SOLR-11228: Exclude static html files in the partials directory from authentication and authorization checks. The UI
will open correctly with kerberos enabled
(Ishan Chattopadhyaya, Varun Thacker)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.10
- Jetty 9.3.14.v20161028
- Upgrade Notes (3)
- Solr contribs map-reduce, morphlines-core and morphlines-cell have been removed.
- JSON Facet API now uses hyper-log-log for numBuckets cardinality calculation and
calculates cardinality before filtering buckets by any mincount greater than 1.
- ZooKeeper dependency has been upgraded from 3.4.6 to 3.4.10.
Detailed Change List
- New Features (36)
- SOLR-9992: Add support for grouping with PointFIelds.
(Cao Manh Dat)
- SOLR-9994: Add support for CollapseQParser with PointFields.
(Varun Thacker, Cao Manh Dat)
- SOLR-10076: Hide keystore and truststore passwords from /admin/info/* outputs.
(Mano Kovacs via Mark Miller)
- SOLR-6736: Adding support for uploading zipped configsets using ConfigSets API
(Varun Rajput, Ishan Chattopadhyaya,
Noble Paul, Anshum Gupta, Gregory Chanan)
- SOLR-10349: Add totalTermFreq support to TermsComponent.
(Shai Erera)
- SOLR-9993: Add support for ExpandComponent with PointFields.
(Cao Manh Dat)
- SOLR-10239: MOVEREPLICA API
(Cao Manh Dat, Noble Paul, shalin)
- SOLR-9936: Allow configuration for recoveryExecutor thread pool size.
(Tim Owen via Mark Miller)
- SOLR-10447: Collections API now supports a LISTALIASES command to return a list of all collection aliases.
(Yago Riveiro, Ishan Chattopadhyaya, Mark Miller, Steve Molloy, Shawn Heisey, Mike Drob, janhoy)
- SOLR-10446: CloudSolrClient can now be initialized using the base URL of a Solr instance instead of
ZooKeeper hosts. This is possible through the use of newly introduced HttpClusterStateProvider.
To fetch a list of collection aliases, this depends on LISTALIASES command, and hence this way of
initializing CloudSolrClient would not work if you have collection aliases on older versions of Solr
server that doesn't support LISTALIASES.
(Ishan Chattopadhyaya, Noble Paul)
- SOLR-10082: Variance and Standard Deviation aggregators for the JSON Facet API.
Example: json.facet={x:"stddev(field1)", y:"variance(field2)"}
(Rustam Hashimov, yonik)
- SOLR-10505: Add multi-field support to TermsComponent when requesting terms' statistics.
(Shai Erera)
- SOLR-10537: SolrJ: Added SolrParams.toLocalParamsString() and ClientUtils.encodeLocalParamVal.
(David Smiley)
- SOLR-10507: Core Admin status command to emit collection details of each core
(noble)
- SOLR-10521: introducing sort=childfield(field) asc for searching by {!parent}
(Mikhail Khludnev)
- SOLR-9596: Add Solr support for SimpleTextCodec, via <codecFactory class="solr.SimpleTextCodecFactory"/>
in solrconfig.xml (per-field specification in the schema is not possible).
(Steve Rowe)
- SOLR-1485: Add payload support with payload() value source and {!payload_score} and {!payload_check}
query parsers.
(Erik Hatcher)
- SOLR-10430: Add ls command to ZkCLI for listing only sub-directories.
(Peter Szantai-Kis via Mark Miller)
- SOLR-10583: JSON Faceting now supports a query time 'join' domain change option
(hossman)
- SOLR-9530: An Update Processor to convert normal update operation to an atomic operations such as
add, set,inc, remove ,set, removeregex
(Amrit Sarkar, noble)
- SOLR-10303: Add date/time Stream Evaluators
(Gethin James, Dennis Gove, Joel Bernstein)
- SOLR-10351: Add analyze Stream Evaluator to support streaming NLP
(Joel Bernstein)
- SOLR-10426: Add shuffle Streaming Expression
(Joel Bernstein)
- SOLR-10274: The search Streaming Expression should work in non-SolrCloud mode
(Joel Bernstein)
- SOLR-10504: Add echo Streaming Expression
(Joel Bernstein)
- SOLR-10516: Add eval Streaming Expression
(Joel Bernstein)
- SOLR-10566: Add timeseries Streaming Expression
(Joel Bernstein)
- SOLR-10559: Add let, get and tuple Streaming Expressions
(Dennis Gove, Joel Bernstein)
- SOLR-10582: Add Correlation Stream Evaluator
(Joel Bernstein)
- SOLR-10536: stats Streaming Expression should work in non-SolrCloud mode
(Joel Bernstein)
- SOLR-10622: Add regress and predict Stream Evaluators
(Joel Bernstein)
- SOLR-10626: Add covariance Stream Evaluator
(Joel Bernstein)
- SOLR-10625: Add convolution Stream Evaluator
(Joel Bernstein)
- SOLR-10638: Add normalize Stream Evaluator
(Joel Bernstein)
- SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd.
(Ishan Chattopadhyaya, janhoy,
Noble Paul, Hrishikesh Gadre)
- SOLR-10292: Adds CartesianProductStream which turns a single tuple with a multi-valued field into N
tuples, one for each value in the multi-valued field.
(Dennis Gove)
- Optimizations (6)
- SOLR-9184: Add a static convenience method ModifiableSolrParams#of(SolrParams) which returns the same
instance if it already is modifiable, otherwise creates a new ModifiableSolrParams instance.
(Jörg Rathlev via Koji)
- SOLR-10499: facet.heatmap is now significantly faster when the docset (base query) matches everything and there are no
deleted docs. It's also faster when the docset matches a small fraction of the index or none.
(David Smiley)
- SOLR-9217: Reduced heap consumption for filter({!join ... score=...})
(Andrey Kudryavtsev, Gopikannan Venugopalsamy via Mikhail Khludnev)
- SOLR-10548: JSON Facet API now uses hyper-log-log++ for determining the number of buckets
when merging requests from a multi-shard distributed request.
(yonik)
- SOLR-10524: Better ZkStateWriter batching
(Cao Manh Dat, Noble Paul, shalin, Scott Blum)
- SOLR-10619: Optimize using cache for DistributedQueue in case of single-consumer
(Cao Manh Dat, Scott Blum)
- Bug Fixes (28)
- SOLR-10281: ADMIN_PATHS is duplicated in two places and inconsistent. This can cause automatic
retries to /admin/metrics handler by the CloudSolrClient.
(shalin)
- SOLR-10108: bin/solr script recursive copy broken
(Erick Erickson)
- SOLR-10362: "Memory Pool not found" error when reporting JVM metrics.
(ab)
- SOLR-10369: bin\solr.cmd delete and healthcheck now works again; fixed continuation chars ^
(Luis Goes via janhoy)
- SOLR-10387: zkTransfer normalizes destination path incorrectly if source is a windows directory
(gopikannan venugopalsamy, Erick Erickson)
- SOLR-10323: fix to SpellingQueryConverter to properly strip out colons in field-specific queries.
(Amrit Sarkar via James Dyer)
- SOLR-10264: Fixes multi-term synonym parsing in ManagedSynonymFilterFactory.
(Jörg Rathlev, Steve Rowe, Christine Poerschke)
- SOLR-8807: fix Spellcheck "collateMaxCollectDocs" parameter to work with queries that have the
CollpasingQParserPlugin applied.
(James Dyer)
- SOLR-10474: TestPointFields.testPointFieldReturn() depends on order of unsorted hits.
(Steve Rowe)
- SOLR-10473: Correct LBHttpSolrClient's confusing SolrServerException message when timeAllowed is exceeded.
(Christine Poerschke)
- SOLR-10047: Mismatched Docvalues segments cause exception in Sorting/Faceting. Solr now uninverts per segment
to avoid such exceptions.
(Keith Laban via shalin)
- SOLR-10472: Fixed uninversion (aka: FieldCache) bugs with the numeric PointField classes, and CurrencyField
(hossman)
- SOLR-5127: Multiple highlight fields and wildcards are now supported e.g. hl.fl=title,text_*
(Sven-S. Porst, Daniel Debray, Simon Endele, Christine Poerschke)
- SOLR-10493: Investigate SolrCloudExampleTest failures.
(Erick Erickson)
- SOLR-10552: JSON Facet API numBuckets was not consistent between distributed and non-distributed requests
when there was a mincount > 1. This has been corrected by changing numBuckets cardinality processing to
ignore mincount > 1 for non-distributed requests.
(yonik)
- SOLR-10520: child.facet.field doubled counts at least when rows>0.
(Dr. Oleg Savrasov via Mikhail Khludnev)
- SOLR-10480: Full pagination in JSON Facet API using offset does not work.
(yonik)
- SOLR-10526: facet.heatmap didn't honor facet exclusions ('ex') for distributed search.
(David Smiley)
- SOLR-10500: nested child docs are adopted by neighbour when several parents come in update/json/docs
(Alexey Suprun,noble via Mikhail Khludnev)
- SOLR-10316: Unloading a core can remove a ZK SolrCore registration entry for the wrong SolrCore.
(Mark Miller)
- SOLR-10588: Prevent redundant core reload on config update
(Mikhail Khludnev)
- SOLR-10549: The new 'large' attribute had been forgotten in /schema/fieldtypes?showDefaults=true
(David Smiley)
- SOLR-10615: requests are suspended until SolrDispatchFilter initialization is completed.
After core container shutdown or severe initialization problem Solr responds with
http stauts 404 Not Found instead of 500 as it was before
(Mikhail Khludnev)
- SOLR-8149: Admin UI - Plugins / Stats - active item is now highlighted
(Labuzov Dmitriy via janhoy)
- SOLR-10630: HttpSolrCall.getAuthCtx().new AuthorizationContext() {...}.getParams()
sometimes throws java.lang.NullPointerException
(hu xiaodong via shalin)
- SOLR-9527: Improve distribution of replicas when restoring a collection
(Hrishikesh Gadre, Stephen Lewis, Rohit, Varun Thacker)
- LUCENE-7821: The classic and flexible query parsers, as well as Solr's
"lucene"/standard query parser, should require " TO " in range queries,
and accept "TO" as endpoints in range queries.
(hossman, Steve Rowe)
- SOLR-10735: Windows script (solr.cmd) didn't work properly with directory containing spaces. Adding quotations
to fix
(Uwe Schindler, janhoy, Tomas Fernandez-Lobbe, Ishan Chattopadhyaya)
- Ref Guide (1)
- SOLR-10758: Modernize the Solr ref guide's Chinese language analysis coverage.
(Steve Rowe)
- Other Changes (29)
- SOLR-9221: Remove Solr contribs: map-reduce, morphlines-core and morphlines-cell.
(Steve Rowe)
- SOLR-10249: Refactor IndexFetcher.doFetch() to return a more detailed result.
(Jeff Miller via David Smiley)
- SOLR-10304: Refactor Document handling out of SolrIndexSearcher into a new class "SolrDocumentFetcher".
Deprecated SolrPluginUtils.docListToSolrDocumentList().
(David Smiley)
- SOLR-10352: bin/solr script now prints warning when available system entropy is lower than 300
(Esther Quansah via
Ishan Chattopadhyaya)
- SOLR-10344: Update Solr default/example and test configs to use WordDelimiterGraphFilterFactory.
(Steve Rowe)
- SOLR-10343: Update Solr default/example and test configs to use SynonymGraphFilterFactory.
(Steve Rowe)
- SOLR-10365: Handle a SolrCoreInitializationException while publishing core state during SolrCore creation
(Ishan Chattopadhyaya)
- SOLR-10357: Enable edismax and standard query parsers to handle the option combination
sow=false / autoGeneratePhraseQueries="true" by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery.
(Steve Rowe)
- SOLR-10147: Admin UI -> Cloud -> Graph: Impossible to see shard state
(Amrit Sarkar, janhoy)
- SOLR-10399: Generalize some internal facet logic to simplify points/non-points field handling
(Adrien Grand, hossman)
- SOLR-7383: New DataImportHandler 'atom' example, replacing broken 'rss' example
(Alexandre Rafalovitch)
- SOLR-9601: Redone DataImportHandler 'tika' example, removing all unused and irrelevant definitions
(Alexandre Rafalovitch)
- SOLR-8906: Make transient core cache pluggable
(Erick Erickson)
- SOLR-9745: print errors from solr.cmd
(Gopikannan Venugopalsamy via Mikhail Khludnev)
- SOLR-10394: Rename getSortWithinGroup to getWithinGroupSort in search.grouping.Command class.
(Judith Silverman, Christine Poerschke)
- SOLR-10440: LBHttpSolrClient.doRequest is now always wrapped in a Mapped Diagnostic Context (MDC).
(Christine Poerschke)
- SOLR-10429: UpdateRequest#getRoutes()should copy the response parser
(noble)
- SOLR-10007: Clean up references to CoreContainer and CoreDescriptors
(Erick Erickson)
- SOLR-10151: Use monotonically incrementing counter for doc ids in TestRecovery.
(Peter Szantai-Kis, Mano Kovacs via Mark Miller)
- SOLR-10514: Upgrade Metrics library to 3.2.2.
(ab)
- SOLR-9386: Upgrade Zookeeper to 3.4.10.
(Shawn Heisey, Steve Rowe)
- SOLR-10519: SolrCLI.atPath cannot handle children that begin with a slash.
(Erick Erickson)
- SOLR-9867: Adding isLoading=true as core status. Fixing start after stop scenario in bin/solr
(Andrey Kudryavtsev, Mikhail Khludnev)
- SOLR-7041: Cutover tests to using 'q.op' and 'df' instead of schema 'defaultOperator' and 'defaultSearchField'
(janhoy)
- SOLR-10601: StreamExpressionParser should handle white space around = in named parameters
(Joel Bernstein)
- SOLR-10614: Static fields have turned to instance's field in SimplePostTool.
Enabled TestSolrCLIRunExample.testTechproductsExample().
(Andrey Kudryavtsev, Mikhail Khludnev)
- SOLR-10522: Revert SpellCheckComponent response format change from SOLR-9972 (rel. 6.5.0). While this
was an improvement for the json "arrntv" format, it caused problems for the default json format.
(James Dyer, reported by Nikita Pchelintsev)
- SOLR-10644: solr.in.sh installed by install script should be writable by solr user
(janhoy)
- SOLR-10729: Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and SpatialTermQueryPrefixTreeFieldType.
Instead, switch to LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField.
(David Smiley)
- Bug Fixes (12)
- SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class.
(Vitezslav Zak, Christine Poerschke)
- SOLR-10416: The JSON output of /admin/metrics is fixed to write the container as a
map (SimpleOrderedMap) instead of an array (NamedList).
(shalin)
- SOLR-10277: On 'downnode', lots of wasteful mutations are done to ZK.
(Joshua Humphries, Scott Blum, Varun Thacker, shalin)
- SOLR-10421: Fix params persistence for solr/contrib/ltr (MinMax|Standard)Normalizer classes.
(Jianxiong Dong, Christine Poerschke)
- SOLR-10404: The fetch() streaming expression wouldn't work if a value included query syntax chars (like :+-).
Fixed, and enhanced the generated query to not pollute the queryCache.
(David Smiley)
- SOLR-10423: Disable graph query production via schema configuration <fieldtype ... enableGraphQueries="false">.
This fixes broken queries for ShingleFilter-containing query-time analyzers when request param sow=false.
(Steve Rowe)
- SOLR-10425: Fix indexed="false" on numeric PointFields
(Tomás Fernández Löbbe, hossman)
- SOLR-10341: SQL AVG function mis-interprets field type.
(Joel Bernstein)
- SOLR-10444: SQL interface does not use client cache.
(Joel Bernstein)
- SOLR-10420: Solr 6.x leaking one SolrZkClient instance per second
(Scott Blum, Cao Manh Dat, Markus Jelsma, Steve Rowe)
- SOLR-10439: The new 'large' attribute had been forgotten in /schema/fields?showDefaults=true
(David Smiley)
- SOLR-10527: edismax with sow=false fails to create dismax-per-term queries when any field is boosted.
(Steve Rowe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (7)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.14.v20161028
- Apache Calcite 1.11.0
Detailed Change List
- Upgrade Notes (2)
- SOLR-10226: JMX metric "avgTimePerRequest" (and the corresponding metric in the metrics API for
each handler) used to be a simple non-decaying average based on total cumulative time and the
number of requests. New Codahale Metrics implementation applies exponential decay to this value,
which heavily biases the average towards the last 5 minutes.
(ab)
- SOLR-8593: Parallel SQL now uses Apache Calcite as its SQL framework. As part of this change
the default aggregation mode has been changed to facet rather than map_reduce. There has also been changes
to the SQL aggregate response and some SQL syntax changes. Consult the documentation for full details.
- New Features (31)
- SOLR-9836: Add ability to recover from leader when index corruption is detected on SolrCore creation.
(Mike Drob via Mark Miller)
- SOLR-9926: Allow passing arbitrary java system properties to zkcli.
(Hrishikesh Gadre via Mark Miller)
- SOLR-9885: Allow pre-startup Solr log management in Solr bin scripts to be disabled.
(Mano Kovacs via Mark Miller)
- SOLR-9481: Authentication and Authorization plugins now work in standalone mode if security.json is placed in
SOLR_HOME on every node. Editing config through API is supported but affects only that one node.
(janhoy)
- SOLR-8029: Added new style APIs and a framework for creating new APIs and mapping old APIs to new
(noble, Steve Rowe, Cassandra Targett, Timothy Potter)
- SOLR-9933: SolrCoreParser now supports configuration of custom SpanQueryBuilder classes.
(Daniel Collins, Christine Poerschke)
- SOLR-7955: Auto create .system collection on first request if it does not exist
(noble)
- SOLR-10087: StreamHandler now supports registering custom streaming expressions from the blob store
(Kevin Risden)
- SOLR-9997: Enable configuring SolrHttpClientBuilder via java system property.
(Hrishikesh Gadre via Mark Miller)
- SOLR-9912: Add facet.excludeTerms parameter support.
(Jonny Marks, David Smiley, Christine Poerschke)
- SOLR-9916: Adds Stream Evaluators to support evaluating values from tuples. Supports boolean,
numeric, and conditional evaluators. BooleanOperations have been removed in preference of
BooleanEvaluators.
(Dennis Gove)
- SOLR-9903: Stop interrupting the update executor on shutdown, it can cause graceful shutdowns to put replicas into Leader
Initiated Recovery among other undesirable things.
(Mark Miller)
- SOLR-8396: Add support for PointFields in Solr
(Ishan Chattopadhyaya, Tomás Fernández Löbbe)
- SOLR-9987: Add support for MultiValued DocValues in PointFields using SortedNumericDocValues
(Tomás Fernández Löbbe)
- SOLR-5944: In-place updates of Numeric DocValues. To leverage this, the _version_ field and the updated
field must both be stored=false, indexed=false, docValues=true.
(Ishan Chattopadhyaya, hossman, noble,
shalin, yonik)
- SOLR-10158: Add support for "preload" option in MMapDirectoryFactory.
(Amrit Sarkar via Uwe Schindler)
- SOLR-10153 & SOLR-10152: The Unified and Postings based highlighters: Add hl.bs.type=SEPARATOR along with new param
hl.bs.separator to break passages by a provided single character.
(Amrit Sarkar, David Smiley)
- SOLR-10156: Add significantTerms Streaming Expression
(Joel Bernstein)
- SOLR-8593: Integrate Apache Calcite into the SQLHandler
(Kevin Risden, Cao Manh Dat, Joel Bernstein)
- SOLR-10146: Added button to the Admin UI 'Collection' tab for deleting an inactive shard
(Amrit Sarkar, janhoy)
- SOLR-9999: Instrument DirectUpdateHandler2. This registers existing statistics under metrics API and adds
more metrics to track the rates of update and delete commands.
(ab)
- SOLR-9986: Implement DatePointField
(Cao Manh Dat, Tomás Fernández Löbbe)
- SOLR-8045: Deploy V2 API at /v2 instead of /solr/v2
(Cao Manh Dat, Noble Paul)
- SOLR-10039: New LatLonPointSpatialField replacement for LatLonType (and some uses of RPT). Multi-value capable
indexed geo lat-lon points, query by rect or circle. Efficient distance sorting/boosting too.
(David Smiley)
- SOLR-10250: CloudSolrClient can now return versions for documents added or deleted when "versions=true" is passed.
However, if there is a leader election while this request is in transit, the versions may not be returned from that
shard.
(Boris Naguet, Ishan Chattopadhyaya)
- SOLR-9045: Make RecoveryStrategy settings configurable.
(Christine Poerschke)
- SOLR-10224: Add disk total and disk free metrics.
(ab)
- SOLR-10085: SQL result set fields should be ordered by the field list
(Joel Bernstein)
- SOLR-10254: significantTerms Streaming Expression should work in non-SolrCloud mode
(Joel Bernstein)
- SOLR-10286: string/text fields may now declare themselves as large="true" in the schema. Large fields are always
lazy loaded and will only take up space in the document cache if the actual value is < 512KB. This option
requires "stored" and must not be multiValued. It's intended for fields that might have very large values so that
they don't get cached in memory.
(David Smiley)
- SOLR-9185: Solr's edismax and "Lucene"/standard query parsers will no longer split on whitespace before sending
terms to analysis, if given the "sow=false" request param ("sow"=>"split on whitespace"). This enables multi-term
source synonyms to match at query-time using SynonymGraphFilterFactory; other analysis components will also now
work at query time, e.g. ShingleFilterFactory. By default, and when the "sow=true" param is specified, these
parsers' behavior remains the same: queries will be split on whitespace before sending individual terms to analysis.
(Steve Rowe)
- Bug Fixes (34)
- SOLR-9976: Fix init bug in SegmentsInfoRequestHandlerTest
(hossman)
- SOLR-9977: Fix config bug in DistribDocExpirationUpdateProcessorTest that allowed false assumptions
about when index version changes
(hossman)
- SOLR-9979: Macro expansion should not be done in shard requests
(Tomás Fernández Löbbe)
- SOLR-9114: NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache
(Cao Manh Dat, Varun Thacker)
- SOLR-10049: Collection deletion leaves behind the snapshot metadata
(Hrishikesh Gadre via yonik)
- SOLR-10120: A SolrCore reload can remove the index from the previous SolrCore during replication index rollover.
(Mark Miller)
- SOLR-10124: Replication can skip removing a temporary index directory in some cases when it should not.
(Mark Miller)
- SOLR-9996: Unstored IntPointField returns Long type
(Ishan Chattopadhyaya)
- SOLR-10104: BlockDirectoryCache release hooks do not work with multiple directories.
(Mike Drob, Mark Miller)
- SOLR-10121: Fix race conditions in HDFS BlockCache that can contribute to corruption in high
concurrency situations.
(yonik)
- SOLR-10141: Upgrade to Caffeine 2.4.0 since v1.0.1 contributed to BlockCache corruption because the
removal listener was called more than once for some items and not at all for other items.
(Ben Manes, yonik)
- SOLR-10114: Reordered delete-by-query causes inconsistenties between shards that have
child documents
(Mano Kovacs, Mihaly Toth, yonik)
- SOLR-10159: When DBQ is reordered with an in-place update, upon whose updated value the DBQ is based
on, the DBQ fails due to excessive caching in DeleteByQueryWrapper
(Ishan Chattopadhyaya)
- SOLR-10020: CoreAdminHandler silently swallows some errors.
(Mike Drob via Erick Erickson)
- SOLR-10063: CoreContainer shutdown has race condition that can cause a hang on shutdown.
(Mark Miller)
- SOLR-10170: ClassCastException in RecoveryStrategy.
(Mark Miller)
- SOLR-9846: Overseer is not always closed after being started.
(Mark Miller)
- SOLR-10168: ShardSplit can fail with NPE in OverseerCollectionMessageHandler#waitForCoreAdminAsyncCallToComplete.
(Mark Miller)
- SOLR-9824: Some bulk update paths could be very slow due to CUSC polling.
(David Smiley, Mark Miller)
- SOLR-10055: Linux installer now renames existing bin/solr.in.* as bin/solr.in.*.orig to make the installed config in
/etc/defaults be the one found by default when launching solr manually.
(janhoy)
- SOLR-10196: ElectionContext#runLeaderProcess can hit NPE on core close.
(Mark Miller)
- SOLR-10225: Fix HDFS BlockCache evictions metric to not count explicit removal
due to a directory close.
(yonik)
- SOLR-10088: Installer script does not put zoo.cfg in SOLR_HOME
(janhoy)
- SOLR-10226: add back "totalTime" metric to all handlers. See also the back-compat note.
(ab)
- SOLR-9838: "inc" atomic update doesn't respect default field value
(hoss, Amrit Sarkar, Ishan Chattopadhyaya)
- SOLR-10269: MetricsHandler JSON output incorrect.
(ab)
- SOLR-10279: The autoAddReplica feature can result in SolrCores being assigned new shards when using
legacyCloud=false and will also fail on a state check when taking over a core registration with a new
core.
(Mark Miller, Hrishikesh Gadre, Patrick Dvorack)
- SOLR-10184: Fix bin/solr so it can run properly on java9
(hossman, Uwe Schindler)
- SOLR-9516: Admin UI (angular) now works with Kerberos, by excluding serving of /solr/libs/* through
SolrDispatchFilter.
(Cassandra Targett, Amrit Sarkar via Ishan Chattopadhyaya)
- SOLR-10302: Solr's zkcli scripts now able to find the metrics libraries, which it couldn't earlier
(kiran, Ishan Chattopadhyaya)
- SOLR-10283: Learning to Rank (LTR) SolrFeature to reject searches with missing efi (External Feature Information) used by fq.
(Christine Poerschke)
- SOLR-10237: Poly-fields should work with subfields that have docValues=true
(Tomás Fernández Löbbe, David Smiley)
- SOLR-10218: The Schema API commands "add-field-type" and "replace-field-type" improperly specify SimilarityFactory params.
(Benjamin Deininger, Troy Mohl, Steve Rowe)
- SOLR-10319: SolrCore "instanceDir" metric not visible in JMX.
(ab)
- Optimizations (5)
- SOLR-9941: Clear the deletes lists at UpdateLog before replaying from log. This prevents redundantly pre-applying
DBQs, during the log replay, to every update in the log as if the DBQs were out of order.
(hossman, Ishan Chattopadhyaya)
- SOLR-9764: All filters that match all documents in the index now share the same memory (DocSet).
(Michael Sun, yonik)
- SOLR-9584: Support Solr being proxied with another endpoint than default /solr, by using relative links
in AdminUI javascripts
(Yun Jie Zhou via janhoy)
- SOLR-10143: PointFields will create IndexOrDocValuesQuery when a field is both, indexed=true and docValues=true
(Tomás Fernández Löbbe)
- SOLR-10273: The field with the longest value (if it exceeds 4K) is moved to be last in the Lucene Document in order
to benefit from stored field optimizations in Lucene that can avoid reading it when it's not needed. If the field is
multi-valued, they all move together to the end to retain order.
(David Smiley)
- Other Changes (28)
- SOLR-9980: Expose configVersion in core admin status
(Jessica Cheng Mallet via Tomás Fernández Löbbe)
- SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON object rather than a list
(Christine Poerschke in response to bug report from Ricky Oktavianus Lazuardy)
- SOLR-9983: Fixing NullPointerException failure by TestManagedSchemaThreadSafety
adding check for Zookeeper session expiration
(Steve Rowe, Mikhail Khludnev)
- SOLR-10043: Reduce logging of pre-start log rotation
(janhoy)
- SOLR-10018: Increase the default hl.maxAnalyzedChars to 51200 for the Unified & Postings Highlighter so that all
highlighters now have this same default.
(David Smiley)
- SOLR-6246: Added tests to check that the changes in LUCENE-7564 and LUCENE-7670
enable AnalyzingInfixSuggester and BlendedInfixSuggester to play nicely with core reload.
SolrSuggester.build() now throws SolrCoreState.CoreIsClosedException when interrupted
by a core reload/shutdown.
(Steve Rowe)
- SOLR-9800: Factor out FacetComponent.newSimpleFacets method.
(Jonny Marks via Christine Poerschke)
- SOLR-9914: SimpleFacets: refactor "contains" check into "SubstringBytesRefFilter" class.
(Jonny Marks, David Smiley, Christine Poerschke)
- SOLR-10072: The test TestSelectiveWeightCreation appears to be unreliable.
(Michael Nilsson via Mark Miller)
- SOLR-10011: Refactor PointField & TrieField to now have a common base class, NumericFieldType. The
TrieField.TrieTypes and PointField.PointTypes are now consolidated to NumericFieldType.NumberType. This
refactoring also fixes a bug whereby PointFields were not using DocValues for range queries for
indexed=false, docValues=true fields.
(Ishan Chattopadhyaya, Tomás Fernández Löbbe)
- SOLR-9890: factor out ShardResultTransformerUtils.[un]marshSortValue methods
(Judith Silverman, Christine Poerschke)
- SOLR-9966: Convert/migrate tests using EasyMock to Mockito
(Cao Manh Dat, Uwe Schindler)
- SOLR-10173: Make HttpShardHandlerFactory.getReplicaListTransformer more extensible.
(Ramsey Haddad via Christine Poerschke)
- SOLR-9842: UpdateRequestProcessors have no way to guarantee the closing of resources used for a request.
(Mark Miller)
- SOLR-9848: Lower solr.cloud.wait-for-updates-with-stale-state-pause back down from 7 seconds.
(Mark Miller)
- SOLR-10020: Cannot reload a core if it fails initialization.
(Mike Drob via Erick Erickson)
- SOLR-9450: The docs/ folder in the binary distribution now contains a single index.html file linking
to the online documentation, reducing the size of the download
(janhoy, Shawn Heisey, Uwe Schindler)
- SOLR-7453: Remove replication & backup scripts in the solr/scripts directory of the checkout
(Varun Thacker)
- SOLR-10214: Remove unused HDFS BlockCache metrics and add storeFails, as well as adding total
counts for lookups, hits, and evictions.
(yonik)
- SOLR-10134: EmbeddedSolrServer responds on Schema API requests
(Robert Alexandersson via Mikhail Khludnev)
- SOLR-10219: re-enable HDFS tests under JDK9
(hossman, Uwe Schindler)
- SOLR-10155: For numeric types facet.contains= and facet.prefix= are now rejected.
(Gus Heck, Christine Poerschke)
- SOLR-10171: Add Constant Reduction Rules to Calcite Planner
(Kevin Risden)
- SOLR-10230: default TTL of PKIAuthenticationPlugin increased to 10secs
(noble)
- SOLR-10235: Fix DIH's TestJdbcDataSource to work with Java 9 and other Java runtimes that
do not use the same DriverManager implementation like Oracle's original one. The test now
uses a fully implemented Driver instance returning a mock connection. The test also works
correct now if other drivers were installed before test execution (e.g., through IDE).
(hossman, Uwe Schindler)
- SOLR-8876: change morphline test config files to work around 'importCommands' bug when using java9
(hossman)
- SOLR-10247: Support non-numeric metrics and a "compact" format of /admin/metrics output.
(ab)
- SOLR-9990: Add PointFields in example/default schemas
(Tomás Fernández Löbbe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.14.v20161028
Detailed Change List
- Bug Fixes (5)
- SOLR-10130, SOLR-10182: Serious performance degradation in Solr 6.4.1 due to the new metrics collection.
Default settings in solrconfig.xml /config/indexConfig/metrics have been changed to turn off
IndexWriter metrics collection. Directory level metrics collection has been completely removed until
a better design is found.
(ab, ishan)
- SOLR-10138: Transaction log replay can hit an NPE due to new Metrics code.
(ab)
- SOLR-10083: Fix instanceof check in ConstDoubleSource.equals
(Pushkar Raste via Christine Poerschke)
- SOLR-10190: Fix NPE in CloudSolrClient when reading stale alias
(Janosch Woschitz via Tomás Fernández Löbbe)
- SOLR-10192: Fix copy/paste in solr-ltr pom.xml template.
(Christine Poerschke)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.14.v20161028
- Bug Fixes (4)
- SOLR-9969: "Plugin/Stats" section of the UI doesn't display empty metric types
(Tomás Fernández Löbbe)
- SOLR-8491: solr.cmd SOLR_SSL_OPTS is overwritten
(Sam Yi, Andy Hind, Marcel Berteler, Kevin Risden)
- SOLR-10031: Validation of filename params in ReplicationHandler
(Hrishikesh Gadre, janhoy)
- SOLR-10054: Core swapping doesn't work with new metrics changes in place
(ab)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.14.v20161028
Detailed Change List
- Upgrade Notes (8)
- SOLR-9166: Export handler returns zero for numeric fields that are not in the original doc. One
consequence of this change is that you must be aware that some tuples will not have values if
there were none in the original document.
- SOLR-8785: Metrics related classes in org.apache.solr.util.stats have been removed in favor of
the dropwizard metrics library. Any custom plugins using these classes should be changed to use
the equivalent classes from the metrics library.
As part of this, the following changes were made to the output of Overseer Status API:
- The "totalTime" metric has been removed because it is no longer supported
- The metrics "75thPctlRequestTime", "95thPctlRequestTime", "99thPctlRequestTime"
and "999thPctlRequestTime" in Overseer Status API have been renamed to "75thPcRequestTime", "95thPcRequestTime"
and so on for consistency with stats output in other parts of Solr.
- The metrics "avgRequestsPerMinute", "5minRateRequestsPerMinute" and "15minRateRequestsPerMinute" have been
replaced by corresponding per-second rates viz. "avgRequestsPerSecond", "5minRateRequestsPerSecond"
and "15minRateRequestsPerSecond" for consistency with stats output in other parts of Solr.
- SOLR-9708: You are encouraged to try out the UnifiedHighlighter by setting hl.method=unified and report feedback. It
might become the default in 7.0. It's more efficient/faster than the other highlighters, especially compared to the
original Highlighter. That said, some options aren't supported yet.
It will get more features in time, especially with your input. See HighlightParams.java
for a listing of highlight parameters annotated with which highlighters use them.
hl.useFastVectorHighlighter is now considered deprecated in lieu of hl.method=fastVector.
- SOLR-9712: maxWarmingSearchers now defaults to 1, and more importantly commits will now block if this
limit is exceeded instead of throwing an exception (a good thing). Consequently there is no longer a
risk in overlapping commits. Nonetheless users should continue to avoid excessive committing. Users are
advised to remove any pre-existing maxWarmingSearchers entries from their solrconfig.xml files.
- SOLR-7466: complexphrase query parser now supports leading wildcards, beware of its' possible heaviness.
Users are encouraged to use ReversedWildcardFilter in index time analysis.
- New Features (40)
- SOLR-9918: Add SkipExistingDocumentsProcessor that skips duplicate inserts and ignores updates to missing docs
(Tim Owen via koji)
- SOLR-9293: Solrj client support for hierarchical clusters and other topics
marker.
(Dawid Weiss)
- SOLR-9681: FacetModule / JSON Facet API added the ability to add filters directly to
any facet command. The filters are applied after any domain change operations.
Example: { type:terms, field:category, filter:"user:yonik" }
(yonik)
- SOLR-9442, SOLR-9787: Adds Array of Name Type Value (json.nl=arrntv) style to JSONResponseWriter.
(Jonny Marks, Christine Poerschke, hossman)
- SOLR-8542: Adds Solr Learning to Rank (LTR) plugin for reranking results with machine learning models.
(Michael Nilsson, Diego Ceccarelli, Joshua Pantony, Jon Dorando, Naveen Santhapuri, Alessandro Benedetti, David Grohmann, Christine Poerschke)
- SOLR-9055: Make collection backup/restore extensible.
(Hrishikesh Gadre, Varun Thacker, Mark Miller)
- SOLR-9682: JSON Facet API: added "param" query type to facet domain filter specification to obtain
filters via query parameters.
(yonik)
- SOLR-9038: Add a command-line tool to manage the snapshots functionality
(Hrishikesh Gadre via yonik)
- SOLR-9633: Limit memory consumed by FastLRUCache with a new 'maxRamMB' config parameter.
(yonik, Michael Sun, shalin)
- SOLR-9666: SolrJ LukeResponse support dynamic fields
(Fengtan via Kevin Risden)
- SOLR-9077: Streaming expressions should support collection alias
(Kevin Risden)
- SOLR-9324: Support Secure Impersonation / Proxy User for solr authentication
(Gregory Chanan, Hrishikesh Gadre via yonik)
- SOLR-9721: javabin Tuple parser for streaming and other end points
(noble)
- SOLR-9708: Added UnifiedSolrHighlighter, a highlighter adapter for Lucene's UnifiedHighlighter. The adapter is a
derivative of the PostingsSolrHighlighter, supporting mostly the same parameters with some differences.
Introduced "hl.method" parameter which can be set to original|fastVector|postings|unified to pick the highlighter at
runtime without the need to modify solrconfig from the default configuration. hl.useFastVectorHighlighter is now
considered deprecated in lieu of hl.method=fastVector.
(Timothy Rodriguez, David Smiley)
- SOLR-9728: Ability to specify Key Store type in solr.in.sh file for SSL
(Michael Suzuki, Kevin Risden)
- SOLR-5043: New solr.dns.prevent.reverse.lookup system property that can be used to prevent long core
(re)load delays on systems with missconfigured hostname/DNS
(hossman)
- SOLR-9844: FieldCache information fetched via the mbeans handler or seen via the UI now displays the total size used.
The individual cache entries in the response are now formatted better as well.
(Varun Thacker)
- SOLR-9513: Generic authentication plugins (GenericHadoopAuthPlugin and ConfigurableInternodeAuthHadoopPlugin) that delegate
all functionality to Hadoop authentication framework.
(Hrishikesh Gadre via Ishan Chattopadhyaya)
- SOLR-9860: Enable configuring invariantParams via HttpSolrClient.Builder
(Hrishikesh Gadre, Ishan Chattopadhyaya)
- SOLR-4735, SOLR-9921: Improve metrics reporting. This uses the dropwizard metric library, adding an internal
API for registering and reporting metrics from Solr components. Several new metrics and an improved JMX
reporter have been added
(Alan Woodward, Jeff Wartes, Christine Poerschke, Kelvin Wong, shalin, ab)
- SOLR-9788: Use instrumented jetty classes provided by the dropwizard metric library.
(shalin)
- SOLR-9805: Use metrics-jvm library to instrument jvm internals such as GC, memory usage and others.
(shalin)
- SOLR-9812, SOLR-9911, SOLR-9960: Added a new /admin/metrics API to return all metrics collected by Solr via API.
API supports four optional multi-valued parameters:
-
'group' (all,jvm,jetty,node,core),
-
'type' (all,counter,timer,gauge,histogram),
-
'prefix' that filters the returned metrics,
-
'registry' that selects one or more registries by prefix (eg. solr.jvm,solr.core.collection1)
-
Example: http://localhost:8983/solr/admin/metrics?group=jvm,jetty&type=counter
-
Example: http://localhost:8983/solr/admin/metrics?group=jvm&prefix=buffers,os
-
Example: http://localhost:8983/solr/admin/metrics?registry=solr.node,solr.core&prefix=ADMIN
(shalin, ab)
- SOLR-9884: Add version to segments handler output
(Steven Bower via Erick Erickson)
- SOLR-9725: Substitute properties into JdbcDataSource configuration
( Jamie Jackson, Yuri Sashevsky via Mikhail Khludnev)
- SOLR-9877: SOLR-9923: SOLR-9948: Use instrumented http client and connection pool in HttpShardHandler and
UpdateShardHandler. The metrics are collected per query-less URL and method by default but it can be configured
to host/method and per-method as well.
(shalin)
- SOLR-9880: Add Ganglia, Graphite and SLF4J metrics reporters.
(ab)
- SOLR-9897: Add hl.requireFieldMatch toggle support when using the UnifiedHighlighter. Defaults to false like the
other highlighters that support this.
(David Smiley)
- SOLR-9905: Add NullStream to isolate the performance of the ExportWriter
(Joel Bernstein)
- SOLR-9891: Add mkroot command to bin/solr and bin/solr.cmd
(Erick Erickson)
- SOLR-9668,SOLR-7197: introduce cursorMark='true' in SolrEntityProcessor
(Yegor Kozlov, Raveendra Yerraguntl via Mikhail Khludnev)
- SOLR-9684: Add priority Streaming Expression
(Joel Bernstein, David Smiley)
- SOLR-9896: Instrument and collect metrics from query, update, core admin and core load thread pools.
(shalin)
- SOLR-9854: Collect metrics for index merges and index store IO
(ab)
- SOLR-8530: Add HavingStream to Streaming API and StreamingExpressions
(Joel Bernstein)
- SOLR-7466: Enable leading wildcard in complexphrase query parser, optimize it with ReversedWildcardFilterFactory
when it's provided
(Mikhail Khludnev)
- SOLR-9935: Add hl.fragsize support when using the UnifiedHighlighter to avoid snippets/Passages that are too small.
Defaults to 70.
(David Smiley)
- SOLR-9856: Collect metrics for shard replication and tlog replay on replicas
(ab).
- SOLR-9886: Add a 'enable' flag to caches to enable/disable them
(Pushkar Raste, noble)
- SOLR-9947: Clean up some SolrInfoMBean categories. Add an alternative hierarchical view in JMX
for SolrInfoMBeans, which uses similar conventions to SolrJmxReporter.
(ab)
- Optimizations (5)
- SOLR-9704: Facet Module / JSON Facet API: Optimize blockChildren facets that have
filters specified by using those filters as acceptDocs.
(yonik)
- SOLR-9726: Reduce number of lookupOrd calls made by the DocValuesFacets.getCounts method.
(Jonny Marks via Christine Poerschke)
- SOLR-9772: Deriving distributed sort values (fieldSortValues) should reuse
comparator and only invalidate leafComparator.
(John Call via yonik)
- SOLR-9786: FieldType has a new getSetQuery() method that can take a set of terms
and create a more efficient query (such as TermsQuery). The solr query parser has been
changed to use this method when appropriate. The parser also knows when it is being
used to parse a filter and will create TermsQueries from large lists of normal terms
or numbers, resulting in a query that will execute faster. This also acts to avoid
BooleanQuery maximum clause limit. Query parsing itself has also been optimized,
resulting in less produced garbage and 5-7% better performance.
(yonik)
- SOLR-9902: StandardDirectoryFactory should use Files API for it's move implementation.
(Mark Miller, Mike Drob)
- Bug Fixes (38)
- SOLR-9701: NPE in export handler when "fl" parameter is omitted.
(Erick Erickson)
- SOLR-9433: SolrCore clean-up logic uses incorrect path to delete dataDir on failure to create a core.
(Evan Sayer, shalin)
- SOLR-9360: Solr script not properly checking SOLR_PID
(Alessandro Benedetti via Erick Erickson)
- SOLR-9716: RecoveryStrategy sends prep recovery command without setting read time out which can cause
replica recovery to hang indefinitely on network partitions.
(Cao Manh Dat, shalin)
- SOLR-9624: In Admin UI, do not attempt to highlight CSV output
(Alexandre Rafalovitch)
- SOLR-9005: In files example, add a guard condition to javascript URP script
(Alexandre Rafalovitch)
- SOLR-9519: JSON Facet API: don't stop at an empty facet bucket if any sub-facets still have a chance
of matching something due to filter exclusions (which can widen the domain again).
(Michael Sun, yonik)
- SOLR-9740: A bug in macro expansion of multi-valued parameters caused non-expanded values
after the first expanded value in the same multi-valued parameter to be dropped.
(Erik Hatcher, yonik)
- SOLR-9751: PreAnalyzedField can cause managed schema corruption.
(Steve Rowe)
- SOLR-9736: Solr resolves the collection name against the first available leader or first replica
of the first slice. This puts undue pressure on leader cores and likely on the wrong ones. This is
fixed to randomly pick a leader on updates or a replica core otherwise.
(Cao Manh Dat via shalin)
- SOLR-9284: The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely.
(Mark Miller, Michael Sun)
- SOLR-9729: JDBCStream improvements
(Kevin Risden)
- SOLR-9626: new Admin UI now also highlights matched terms in the Analysis screen.
(Alexandre Rafalovitch)
- SOLR-9512: CloudSolrClient's cluster state cache can break direct updates to leaders
(noble)
- SOLR-5260: Facet search on a docvalue field in a multi shard collection
(Trym Møller, Erick Erickson)
- SOLR-9768: RecordingJsonParser produces incomplete json
(Wojciech Stryszyk via ab)
- SOLR-9616: Solr throws exception when expand=true on empty index
(Timo Hund via Ishan Chattopadhyaya)
- SOLR-9832: Schema modifications are not immediately visible on the coordinating node.
(Steve Rowe)
- SOLR-9834: A variety of spots in the code can create a collection zk node after the collection has been
removed.
(Mark Miller)
- SOLR-9707: Don't forward DeleteByQuery requests to down replicas.
(Jessica Cheng Mallet via Varun Thacker)
- SOLR-9823: CoreContainer incorrectly setting MDCLoggingContext for core
(Jessica Cheng Mallet via Erick Erickson)
- SOLR-1953: It may be possible for temporary files to accumulate until the Solr process is shut down.
(Karl Wright, Mark Miller)
- SOLR-9847: Stop blocking further schema updates while waiting for a pending update to propagate to other replicas.
This reduces the likelihood of a (time-limited) distributed deadlock during concurrent schema updates.
(Mark Miller, Steve Rowe)
- SOLR-9760: Windows script doesn't need write permission
(Alex Crome by Mikhail Khludnev)
- SOLR-9699,SOLR-4668: fix exception from core status in parallel with core reload
(Mikhail Khludnev)
- SOLR-9859: replication.properties cannot be updated after being written and neither replication.properties or
index.properties are durable in the face of a crash.
(Pushkar Raste, Chris de Kok, Cao Manh Dat, Mark Miller)
- SOLR-9901: Implement move in HdfsDirectoryFactory.
(Mark Miller)
- SOLR-9900: fix false positives on range queries with ReversedWildcardFilterFactory
(Yonik Seeley via Mikhail Khludnev)
- SOLR-9495: AIOBE with confusing message for incomplete sort spec in Streaming Expression
(Gus Heck, Joel Bernstein)
- SOLR-9154: Fix DirectSolrSpellChecker to work when added through the Config API.
(Anshum Gupta)
- SOLR-9919: random Streaming Expression is not registered in /stream or /graph handler
(Joel Bernstein)
- SOLR-7495: Support Facet.field on a non-DocValued, single-value, int field
(Varun Thacker, Scott Stults)
- SOLR-9917: JSON Facet API percentile function caused a NullPointerException in distributed mode when
there were no values in a bucket from a shard.
(yonik)
- SOLR-9931: JSON Facet API hll (hyper-log-log) function returned 0 for non-empty buckets with no field values
in local mode, but nothing for distributed mode. Both modes now return 0.
(yonik)
- SOLR-9503: NPE in Replica Placement Rules when using Overseer Role with other rules
(Tim Owen via noble)
- SOLR-9883: Example schemaless solr config files can lead to invalid tlog replays: when updates are buffered,
update processors ordered before DistributedUpdateProcessor, e.g. field normalization, are never run.
(Steve Rowe)
- SOLR-9644: SimpleMLTQParser and CloudMLTQParser did not handle field boosts properly
and CloudMLTQParser included extra strings from the field definitions in the query.
(Ere Maijala via Anshum Gupta)
- SOLR-9954: Prevent against failure during failed snapshot cleanup from swallowing the actual cause
for the snapshot to fail.
(thelabdude)
- Other Changes (28)
- SOLR-7539: Upgrade the clustering plugin to Carrot2 3.15.0.
(Dawid Weiss)
- SOLR-9621: Remove several Guava & Apache Commons calls in favor of java 8 alternatives.
(Michael Braun via David Smiley)
- SOLR-9720: Refactor Responsewriters to remove dependencies on TupleStream,
Tuple, Explanation
(noble)
- SOLR-9717: Refactor '/export' to not hardcode the JSON output and to use an API
(noble)
- SOLR-9739: JavabinCodec implements PushWriter interface
(noble)
- SOLR-8332: Factor HttpShardHandler[Factory]'s url shuffling out into a ReplicaListTransformer class.
(Christine Poerschke, Noble Paul)
- SOLR-9597: Add setReadOnly(String ...) to ConnectionImpl
(Kevin Risden)
- SOLR-9609: Change hard-coded keysize from 512 to 1024
(Jeremy Martini via Erick Erickson)
- SOLR-8785: Use Dropwizard Metrics library for core metrics. The copied over code in
org.apache.solr.util.stats has been removed.
(Jeff Wartes, Kelvin Wong, Christine Poerschke, shalin)
- SOLR-9784: Refactor CloudSolrClient to eliminate direct dependency on ZK
(noble)
- SOLR-9801: Upgrade jetty to 9.3.14.v20161028
(shalin)
- SOLR-9783: (Search|Top)Group[s]ShardResponseProcessor.process: turned sortWithinGroup null check into assert.
(Christine Poerschke)
- SOLR-9660: in GroupingSpecification factor [group](sort|offset|limit) into [group](sortSpec)
(Judith Silverman, Christine Poerschke)
- SOLR-9819: Upgrade commons-fileupload to 1.3.2, fixing a potential vulnerability CVE-2016-3092
(Anshum Gupta)
- SOLR-9827: ConcurrentUpdateSolrClient creates a RemoteSolrException if the remote host responded with a non-ok
response (instead of a SolrException) and includes the remote error message as part of the exception message
(Tomás Fernández Löbbe)
- SOLR-9846: OverseerAutoReplicaFailoverThread can take too long to stop and leak out of unit tests.
(Mark Miller)
- SOLR-8959: Refactored TestSegmentSorting out of TestMiniSolrCloudCluster
(hossman)
- SOLR-9874: Solr will reject CREATEALIAS requests if target collections don't exist
(Tomás Fernández Löbbe)
- SOLR-9878: fixing lazy logic for retrieving ReversedWildcardFilterFactory in SolrQueryParserBase
(Mikhail Khludnev)
- SOLR-9758: refactor preferLocalShards implementation
(Christine Poerschke)
- SOLR-9448: providing a test to workaround a differently named uniqueKey field
(Mikhail Khludnev)
- SOLR-9899: StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.
(Mark Miller)
- SOLR-9915: PeerSync alreadyInSync check is not backwards compatible and results in full replication during a rolling restart
(Tim Owen via noble)
- SOLR-3990: Moves getIndexSize() from ReplicationHandler to SolrCore
(Shawn Heisey)
- SOLR-9944: Map the nodes function name to the GatherNodesStream
(Joel Bernstein)
- SOLR-9777: IndexFingerprinting should use getCombinedCoreAndDeletesKey() instead of getCoreCacheKey() for per-segment caching
(Ishan Chattopadhyaya)
- SOLR-9934: SolrTestCase.clearIndex has been improved to take advantage of low level test specific logic that
clears the index metadata more completely then a normal *:* DBQ can due to update versioning.
(hossman)
- SOLR-9893: Update Mockito to version 2.6.2 for Java 9 support. Disable all legacy EasyMock tests when running
on Java 9 until they were migrated to Mockito.
(Uwe Schindler)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.12.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.8.v20160314
Detailed Change List
- Upgrade Notes (6)
- If you use the JSON Facet API (json.facet) with method=stream, you must now set sort='index asc' to get the streaming
behavior; otherwise it won't stream. Reminder: "method" is a hint that doesn't change defaults of other parameters.
- If you use the JSON Facet API (json.facet) to facet on a numeric field and if you use mincount=0 or if you set the
prefix, then you will now get an error as these options are incompatible with numeric faceting.
- Solr's logging verbosity at the INFO level has been greatly reduced, and
you may need to update the log configs to use the DEBUG level to see all the
logging messages you used to see at INFO level before.
- We are no longer backing up solr.log and solr_gc.log files in date-stamped copies forever. If you relied on
the solr_log_<date> or solr_gc_log_<date> being in the logs folder that will no longer be the case.
See SOLR-9570 for details.
- The create/deleteCollection methods on MiniSolrCloudCluster have been
deprecated. Clients should instead use the CollectionAdminRequest API. In
addition, MiniSolrCloudCluster#uploadConfigDir(File, String) has been
deprecated in favour of #uploadConfigSet(Path, String)
- The bin/solr.in.sh (bin/solr.in.cmd on Windows) is now completely commented by default. Previously, this wasn't so,
which had the effect of masking existing environment variables.
- New Features (27)
- SOLR-5725: facet.method=enum can bypass exact counts calculation with facet.exists=true, it just returns 1 for
terms which exists in result docset.
(Alexey Kozhemiakin, Sebastian Koziel, Radoslaw Zielinski via Mikhail Khludnev)
- SOLR-9127: Excel workbook (.xlsx) response writer. use 'wt=xlsx'
(Tony Moriarty, noble)
- SOLR-9469: JettySolrRunner now has the option of restarting using a different
port
(Alan Woodward)
- SOLR-9319: DELETEREPLICA can accept a 'count' and remove appropriate replicas
(Nitin Sharma, noble)
- SOLR-8186: Reduce logging to logs/solr-<port>-console.log when not running in foreground mode
Show timestamp also in foreground log. Also removes some logging noise.
(janhoy)
- SOLR-8487: Adds CommitStream to support sending commits to a collection being updated.
(Dennis Gove)
- SOLR-9534: You can now set Solr's log level through environment variable SOLR_LOG_LEVEL
Also adds conveience arguments -q (quiet: WARN) and -v (verbose: DEBUG) to bin/solr
(janhoy)
- SOLR-9537: Support facet scoring with the scoreNodes expression
(Joel Bernstein)
- SOLR-9558: DIH TemplateTransformerto to support multivalued fields
(Ted Sullivan via noble)
- SOLR-9557: Every implicit requesthandler now has a default 'useParams' attribute
(noble)
- SOLR-9572: config API to show expanded useParams for request handlers inline
(noble)
- SOLR-9258: Optimizing, storing and deploying AI models with Streaming Expressions
(Cao Manh Dat, Joel Bernstein)
- SOLR-9205: Added method LukeResponse.getSchemaFlags() which returns field
information as an EnumSet
(Fengtan, Alan Woodward)
- SOLR-9520: Kerberos delegation support in SolrJ
(Ishan Chattopadhyaya, noble)
- SOLR-9146: Parallel SQL engine should support >, >=, <, <=, <>, != syntax
(Timothy Potter, Joel Bernstein, Kevin Risden)
- SOLR-9337: Add fetch Streaming Expression
(Joel Bernstein)
- SOLR-9103: Restore ability for users to add custom Streaming Expressions
(Cao Manh Dat)
- SOLR-9657: New TemplateUpdateProcessorFactory added
(noble)
- SOLR-9417: Allow daemons to terminate when they finish iterating a topic
(Joel Bernstein)
- SOLR-8370: Display configured Similarity in Schema-Browser, both global/default and per-field/field-type
(janhoy, Alexandre Rafalovitch)
- SOLR-9326: Ability to create/delete/list snapshots at collection level.
(Hrishikesh Gadre via yonik)
- SOLR-9662: New parameter -u <user:pass> in bin/post to pass basicauth credentials
(janhoy)
- SOLR-9654: Add "overrequest" parameter to JSON Facet API to control amount of overrequest
on a distributed terms facet.
(yonik)
- SOLR-2212: Add a factory class corresponding to Lucene's NoMergePolicy.
(Lance Norskog, Cao Manh Dat via shalin)
- SOLR-9670: Support SOLR_AUTHENTICATION_OPTS in solr.cmd
(janhoy)
- SOLR-9559: Add ExecutorStream to execute stored Streaming Expressions
(Joel Bernstein)
- SOLR-1085: Add support for MoreLikeThis queries and responses in SolrJ client.
(Maurice Jumelet, Bill Mitchell, Cao Manh Dat via shalin)
- Bug Fixes (26)
- SOLR-9310: PeerSync fails on a node restart due to IndexFingerPrint mismatch
(Pushkar Raste, noble)
- SOLR-9484: The modify collection API should wait for the modified properties to show up in the
cluster state.
(Cao Manh Dat, shalin)
- SOLR-9507: CoreContainer threads now correctly set their MDC logging values
(Alan Woodward)
- SOLR-9522: Improve error handling in ZKPropertiesWriter
(Varun Thacker)
- SOLR-8080: bin/solr start script now exits with informative message if using wrong Java version
(janhoy)
- SOLR-9475: bin/install_solr_service.sh script got improved detection of Linux distro, especially within
virtualized/Docker environment through parsing of /etc/*-release files. Now also supports CentOS.
(janhoy)
- SOLR-9524: SolrIndexSearcher.getIndexFingerprint uses dubious synchronization
(Mike Drob, noble)
- SOLR-9542: Kerberos delegation tokens requires Jackson library
(Ishan Chattopadhyaya via noble)
- SOLR-9330: Fix AlreadyClosedException on admin/mbeans?stats=true
(Mikhail Khludnev)
- SOLR-9411: Better validation for Schema API add-field and add-dynamic-field
(janhoy, Steve Rowe)
- SOLR-9504: A replica with an empty index becomes the leader even when other more qualified replicas
are in line.
(shalin)
- SOLR-9554: Fix NullPointerException when cores are loaded in parallel and switch schema.xml to managed-scheme.
(Alan Woodward, Mikhail Khludnev)
- SOLR-9556: OverseerAutoFailoverReplicaThread was not exiting on interrupt
(Alan Woodward)
- SOLR-9563: Collection creation could fail if an ADDREPLICA subrequest arrived
at a node before its local state had updated with the new collection data
(Alan Woodward)
- SOLR-9278: Index replication interactions with IndexWriter can cause deadlock.
(Xunlong via Mark Miller)
- SOLR-9604: Pooled SSL connections were not being re-used
(Alan Woodward,
Mikhail Khludnev, hossman)
- SOLR-9325: solr.log is now written to $SOLR_LOGS_DIR without changing log4j.properties
(janhoy)
- SOLR-9518: Kerberos Delegation Tokens don't work without a chrooted ZK
(Ishan Chattopadhyaya,via noble)
- SOLR-9687: Fixed Interval Facet count issue in cases of open/close intervals on the same values
(Andy Chillrud, Tomás Fernández Löbbe)
- SOLR-9441: Solr collection backup on HDFS can only be manipulated by the Solr process owner.
(Hrishikesh Gadre via Mark Miller)
- SOLR-9536: OldBackupDirectory timestamp field needs to be initialized to avoid NPE.
(Hrishikesh Gadre, hossman via Mark Miller)
- SOLR-2039: Multivalued fields with dynamic names does not work properly with DIH.
(K A, ruslan.shv, Cao Manh Dat via shalin)
- SOLR-4164: group.limit=-1 was not supported for grouping in distributed mode.
(Cao Manh Dat, Lance Norskog, Webster Homer, hossman, yonik)
- SOLR-9692: blockUnknown property makes inter-node communication impossible
(noble)
- SOLR-2094: XPathEntityProcessor should reinitialize the XPathRecordReader instance if
the 'forEach' or 'xpath' attributes are templates & it is not a root entity
(Cao Manh Dat, noble)
- SOLR-9697: zk upconfig broken on windows
(Xavier Jmlucjav via janhoy)
- Optimizations (11)
- SOLR-9374: Speed up Jmx MBean retrieval for FieldCache.
(Tim Owen via shalin)
- SOLR-9449: Example schemas do not index _version_ field anymore because the field
has DocValues enabled already.
(shalin)
- SOLR-9447: Do not clone SolrInputDocument if update processor chain does not contain custom processors.
(shalin)
- SOLR-9452: JsonRecordReader should not deep copy document before handler.handle().
(noble, shalin)
- SOLR-9142: JSON Facet API: new method=dvhash can be chosen for fields with high cardinality.
(David Smiley)
- SOLR-9446: Leader failure after creating a freshly replicated index can send nodes into recovery even if
index was not changed
(Pushkar Raste, noble)
- SOLR-9592: retrieving docValues as stored values was sped up by using the proper leaf
reader rather than ask for a global view. In extreme cases, this leads to a 100x speedup.
(Takahiro Ishikawa, yonik)
- SOLR-9566: Don't put replicas into recovery when first creating a Collection
(Alan Woodward)
- SOLR-9546: Eliminate unnecessary boxing/unboxing going on in SolrParams
(Pushkar Raste, noble)
- SOLR-9506: cache IndexFingerprint for each segment
(Pushkar Raste, yonik, noble)
- SOLR-7506: Roll over GC logs by default via bin/solr scripts
(shalin, janhoy)
- Other Changes (44)
- SOLR-9412: Add failOnMissingParams option to MacroExpander, add TestMacroExpander class.
(Jon Dorando, Christine Poerschke)
- SOLR-9406: SolrSuggester should selectively register close hook
(Gethin James, Joel Bernstein)
- SOLR-8961: Add a test module for solr-test-framework
(Alan Woodward)
- SOLR-9474: MiniSolrCloudCluster will not reuse ports by default when
restarting its JettySolrRunners
(Alan Woodward)
- SOLR-9498: Remove HDFS properties from DIH solrconfig.xml, as started in SOLR-6943
(Alexandre Rafalovitch)
- SOLR-9365: Reduce noise in solr logs during graceful shutdown.
(Cao Manh Dat via shalin)
- SOLR-9451: Make clusterstatus command logging less verbose.
(Varun Thacker)
- SOLR-9502: ResponseWriters should natively support MapSerializable
(noble)
- SOLR-9538: Relocate (BinaryResponse|JSON|Smile)Writer tests to org.apache.solr.response
which is the package of the classes they test.
(Jonny Marks via Christine Poerschke)
- SOLR-9508: Install script install_solr_service.sh now checks existence of tools.
New option -n to avoid starting service after installation
(janhoy)
- SOLR-7826: Refuse "bin/solr create" if run as root, unless -force is specified
(janhoy, Binoy Dalal)
- SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target
"generate-website-quickstart" to convert the bundled version of the tutorial into one suitable
for the website.
- SOLR-5563: Move lots of SolrCloud logging from 'info' to 'debug'
(janhoy, Alan
Woodward)
- SOLR-9544: Allow ObjectReleaseTracker more time to check for asynchronously
closing resources
(Alan Woodward)
- SOLR-6677: Reduced logging during startup and shutdown, moved more logs to DEBUG level
(janhoy, Shawn Heisey, Alan Woodward)
- SOLR-6090: Remove unreachable printLayout usage in cloud tests.
(Cao Manh Dat via shalin)
- SOLR-9551: Add JSONWriter constructor variant, JSONWriterTest.testConstantsUnchanged test.
(Jonny Marks, Christine Poerschke)
- SOLR-9500: Add a LogLevel annotation to set log levels on specific tests
(Alan
Woodward)
- SOLR-9548: The beginning of solr.log now starts with a more informative welcome message
(janhoy)
- SOLR-9547: Do not allow bin/solr start as root user, unless -force param specified
(janhoy)
- SOLR-9567: Make ReRankQParserPlugin's private ReRankCollector a public class of its own.
(Christine Poerschke)
- SOLR-7436: Solr stops printing stacktraces in log and output
(janhoy, hossman, Markus Jelsma)
- SOLR-9576: Make FieldAnalysisRequestHandler, DocumentAnalysisRequestHandler & DumpRequestHandler
implicit
(noble)
- SOLR-9574: Factor out AbstractReRankQuery from ReRankQParserPlugin's private ReRankQuery.
(Christine Poerschke)
- SOLR-5041: Add a test to make sure that a leader always recovers from log on startup.
(Cao Manh Dat, shalin)
- SOLR-9588: Remove Guava dependency from SolrJ
(Ishan Chattopadhyaya, noble)
- SOLR-8140: Remove mentions of unimplemented admin-extra from the new Admin UI
(Alexandre Rafalovitch)
- SOLR-9589: Remove jackson dependency from SolrJ
(Ishan Chattopadhyaya, noble)
- SOLR-8385: Narrow StreamFactory.withFunctionName clazz parameter to prevent misconfiguration
(Jason Gerlowski, Kevin Risden)
- SOLR-8969: SQLHandler causes NPE in non-cloud mode
(Markus Jelsma, Kevin Risden)
- SOLR-9610: New AssertTool in SolrCLI for easier cross platform assertions from command line
(janhoy)
- SOLR-9680: Better error messages in SolrCLI when authentication required
(janhoy)
- SOLR-9639: Test only fix. Prevent CDCR tests from removing collection during recovery that used to blow up jvm
(Mikhail Khludnev)
- SOLR-9625: Add HelloWorldSolrCloudTestCase class
(Christine Poerschke, Alan Woodward, Alexandre Rafalovitch)
- SOLR-9642: Refactor the core level snapshot cleanup mechanism to rely on Lucene
(Hrishikesh Gadre via yonik)
- SOLR-9627: Add QParser.getSortSpec, deprecate misleadingly named QParser.getSort
(Judith Silverman, Christine Poerschke)
- SOLR-9632: Add MiniSolrCloudCluster#deleteAllCollections() method
(Alan Woodward)
- SOLR-9634: Deprecate collection methods on MiniSolrCloudCluster
(Alan Woodward)
- SOLR-7850: Moved defaults within bin/solr.in.sh (and bin/solr.in.cmd on Windows) to bin/solr (and bin/solr.cmd)
such that the default state of these files is to set nothing. This makes Solr work better with Docker.
(David Smiley)
- SOLR-9570: Various log tidying now happens at Solr startup:
Old solr_log_<date> and solr_gc_log_<date> files are removed, avoiding disks to fill up,
solr.log.X files are rotated, preserving solr.log from last run in solr.log.1, solr.log.1 => solr.log.2 etc
solr-*-console.log files are moved into $SOLR_LOGS_DIR/archived/ instead of being overwritten
Last JVM garbage collection log solr_gc.log is moved into $SOLR_LOGS_DIR/archived/
(janhoy)
- SOLR-4531: Add tests to ensure that recovery does not fail on corrupted tlogs.
(Simon Scofield, Cao Manh Dat via shalin)
- SOLR-5245: Add a test to ensure that election contexts are keyed off both collection name and coreNodeName
so that killing a shard in one collection does not result in leader election in a different collection.
See SOLR-5243 for the related bug.
(Cao Manh Dat via shalin)
- SOLR-9533: Reload core config when a core is reloaded
(Gethin James, Joel Bernstein)
- SOLR-9371: Fix bin/solr calculations for start/stop wait time and RMI_PORT.
(Shawn Heisey via Erick Erickson)
- Bug Fixes (15)
- SOLR-9494: Use of {!collapse} sometimes doesn't correctly return true for Collector.needsScores(), especially when the
query was cached. This can cause an exception when 'q' is a SpanQuery or potentially others.
(David Smiley)
- SOLR-6744: fl renaming / alias of uniqueKey field generates null pointer exception in SolrCloud configuration
(Mike Drob via Tomás Fernández Löbbe)
- SOLR-9445: Admin requests are retried by CloudSolrClient and LBHttpSolrClient on failure.
(shalin)
- SOLR-9439: Shard split clean up logic for older failed splits is faulty. The delete shard API
has also been made more resilient against failures resulting from non-existent cores.
(shalin)
- SOLR-9430: Fix locale lookup in DIH <propertyWriter/> to use BCP47 language tags
to be consistent with other places in Solr. Language names still work for backwards
compatibility.
(Uwe Schindler, Boris Steiner)
- SOLR-9389: HDFS Transaction logs stay open for writes which leaks Xceivers.
(Tim Owen via Mark Miller)
- SOLR-9188: blockUnknown property makes inter-node communication impossible
(noble)
- SOLR-9455: Deleting a sub-shard in recovery state can mark parent shard as inactive.
(shalin)
- SOLR-9461: DELETENODE, REPLACENODE should pass down the 'async' param to subcommands
(shalin, noble)
- SOLR-9444: Fix path usage for cloud backup/restore.
(Hrishikesh Gadre, Uwe Schindler, Varun Thacker)
- SOLR-9381: Snitch for freedisk uses '/' instead of 'coreRootDirectory'
(Tim Owen, noble)
- SOLR-9488: Shard split can fail to write commit data on shutdown/restart causing replicas to recover
without replicating the index. This can cause data loss.
(shalin)
- SOLR-9490: Fixed bugs in BoolField that caused it to erroneously return "false" for all docs depending
on usage
(Colvin Cowie, Dan Fox, hossman)
- SOLR-9438: Shard split can be marked successful and sub-shard states switched to 'active' even when
one or more sub-shards replicas do not recover due to the leader crashing or restarting between the time
the replicas are created and before they can recover. This can cause data loss.
(shalin)
- SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to a commit. SolrCloud replication relies on this.
(Jessica Cheng Mallet via Varun Thacker)
- Other Changes (1)
- SOLR-7362: Fix TestReqParamsAPI test failures
(noble, Varun Thacker)
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.12.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.8.v20160314
Detailed Change List
- New Features (20)
- SOLR-9187: Support dates and booleans in /export handler, support boolean DocValues fields
(Erick Erickson)
- SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh
(noble)
- SOLR-7374: Core level Backup/Restore now supports specifying the directory implementation to use
via the "repository" parameter.
(Hrishikesh Gadre, Varun Thacker, Mark Miller)
- SOLR-9216: Support collection.configName in MODIFYCOLLECTION request
(Keith Laban, noble)
- SOLR-9251: Support for a new tag 'role' in replica placement rules
(noble)
- SOLR-9194: Enhance the bin/solr script to perform file operations to/from Zookeeper
(Erick Erickson, janhoy)
- SOLR-9242: Collection Backup/Restore now supports specifying the directory implementation to use
via the "repository" parameter.
(Hrishikesh Gadre, Varun Thacker)
- SOLR-9193: Add scoreNodes Streaming Expression
(Joel Bernstein)
- SOLR-9243: Add terms.list parameter to the TermsComponent to fetch the docFreq for a list of terms
(Joel Bernstein)
- SOLR-9090: Add directUpdatesToLeadersOnly flag to solrj CloudSolrClient.
(Marvin Justice, Christine Poerschke)
- SOLR-9270: Allow spatialContextFactory to be simply "JTS". And if any spatial params include the old
Spatial4j package "com.spatial4j.core" it is rewritten to "org.locationtech.spatial4j" with a warning.
(David Smiley)
- SOLR-9240: Support parallel ETL with the topic expression
(Joel Bernstein)
- SOLR-9275: XML QueryParser support (defType=xmlparser) now extensible via configuration.
(Christine Poerschke)
- SOLR-9200: Add Delegation Token Support to Solr.
(Gregory Chanan)
- SOLR-9038: Solr core snapshots: The current commit can be snapshotted which retains the commit and associates it with
a name. The core admin API can create snapshots, list them, and delete them. Snapshot names can be referenced in
doing a core backup, and in replication. Snapshot metadata is stored in a new snapshot_metadata/ dir.
(Hrishikesh Gadre via David Smiley)
- SOLR-9279: New boolean comparison function queries comparing numeric arguments: gt, gte, lt, lte, eq
(Doug Turnbull, David Smiley)
- SOLR-9252: Feature selection and logistic regression on text
(Cao Manh Dat, Joel Bernstein)
- SOLR-6465: CDCR: fall back to whole-index replication when tlogs are insufficient.
(Noble Paul, Renaud Delbru, shalin)
- SOLR-9320: A REPLACENODE command to decommission an existing node with another new node
(noble, Nitin Sharma, Varun Thacker)
- SOLR-9318: A DELETENODE command to delete all replicas in that node
(noble, Nitin Sharma, Varun Thacker)
- Bug Fixes (40)
- SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed
(Scott Blum, Noble Paul)
- SOLR-9199: ZkController#publishAndWaitForDownStates logic is inefficient
(Hrishikesh Gadre)
- SOLR-9161: Change SolrPluginUtils.invokeSetters implementation to accommodate setter variants.
(Christine Poerschke, Steve Rowe, Uwe Schindler)
- SOLR-9234: srcField parameter works only when all fields are captured in the /update/json/docs
endpoint
(noble)
- SOLR-8546: SOLR-8546: TestLazyCores is failing a lot on the Jenkins cluster.
(Erick Erickson)
- SOLR-9237: DefaultSolrHighlighter.doHighlightingByFastVectorHighlighter can't be overidden
(janhoy)
- SOLR-8626: 404 error when clicking nodes in cloud graph view in angular UI.
(janhoy, Trey Grainger via shalin)
- SOLR-9254: GraphTermsQueryQParserPlugin throws NPE when field being searched is not present in segment
(Joel Bernstein)
- SOLR-8657: Fix SolrRequestInfo error logs if QuerySenderListener is being used
(Pascal Chollet,
Tomás Fernández Löbbe)
- SOLR-8777: Duplicate Solr process can cripple a running process.
(Jessica Cheng Mallet, Scott Blum, shalin)
- SOLR-9246: If the JDBCStream sees an unknown column type it will now throw a detailed exception.
(Dennis Gove)
- SOLR-9181: Fix some races in CollectionStateWatcher API
(Alan Woodward, Scott
Blum)
- SOLR-9235: Fixed NPE when using non-numeric range query in deleteByQuery
(hossman)
- SOLR-9088: Fixed TestManagedSchemaAPI failures which exposed race conditions in the schema API
( Varun Thacker, noble)
- SOLR-9207: PeerSync recovery failes if number of updates requested is high. A new useRangeVersions config option
is introduced (defaults to true) to send version ranges instead of individual versions for peer sync.
(Pushkar Raste, shalin)
- SOLR-8858: SolrIndexSearcher#doc() completely ignores field filters unless lazy field loading is enabled.
(Caleb Rackliffe, David Smiley, shalin)
- SOLR-9236: AutoAddReplicas will append an extra /tlog to the update log location on replica failover.
(Eungsop Yoo, Mark Miller)
- SOLR-9291: ZkSolrResourceLoader should not retry fetching resources if the server has been shutdown.
(shalin)
- SOLR-9287: Including 'score' in the 'fl' param when doing an RTG no longer causes an NPE
(hossman, Ishan Chattopadhyaya)
- SOLR-7280: In cloud-mode sort the cores smartly before loading & limit threads to improve cluster stability
(noble, Erick Erickson, shalin)
- SOLR-9285: Fixed AIOOBE when using ValueSourceAugmenter in single node RTG
(hossman)
- SOLR-9288: Fix [docid] transformer to return -1 when used in RTG with uncommitted doc
(hossman)
- SOLR-9309: Fix SolrCloud RTG response structure when multi ids requested but only 1 found
(hossman)
- SOLR-9334: CloudSolrClient.collectionStateCache is unbounded
(noble)
- SOLR-9339: NPE in CloudSolrClient when the response is null
(noble)
- SOLR-8596: Web UI doesn't correctly generate queries which include local parameters
(Alexandre Rafalovitch, janhoy)
- SOLR-8645: managed-schema is now syntax highlighted in cloud->Tree view
(Alexandre Rafalovitch via janhoy)
- SOLR-8379: UI Cloud->Tree view now shows .txt files correctly
(Alexandre Rafalovitch via janhoy)
- SOLR-9003: New Admin UI's Dataimport screen now correctly displays DIH Debug output
(Alexandre Rafalovitch)
- SOLR-9308: Fix distributed RTG to forward request params, fixes fq and non-default fl params
(hossman)
- SOLR-9179: NPE in IndexSchema using IBM JDK
(noble, Colvin Cowie)
- SOLR-9397: Config API does not support adding caches
(noble)
- SOLR-9405: ConcurrentModificationException in ZkStateReader.getStateWatchers.
(Alan Woodward, Edward Ribeiro, shalin)
- SOLR-9232: Admin UI now fully implements Swap Cores interface
(Alexandre Rafalovitch)
- SOLR-8715: Admin UI's Schema screen now works for fields with stored=false and some content indexed
(Alexandre Rafalovitch)
- SOLR-8911: In Admin UI, enable scrolling for overflowing Versions and JVM property values
(Alexandre Rafalovitch)
- SOLR-9002: Admin UI now correctly displays json and text files in the collection/Files screen
(Upayavira, Alexandre Rafalovitch)
- SOLR-8993: Admin UI now correctly supports multiple DIH handler end-points
(Upayavira, Alexandre Rafalovitch)
- SOLR-9032: Admin UI now correctly implements Create Alias command
(Upayavira, Alexandre Rafalovitch)
- SOLR-9391: LBHttpSolrClient.request now correctly returns Rsp.server when
previously skipped servers were successfully tried.
(Christine Poerschke)
- Optimizations (4)
- SOLR-9219: Make hdfs blockcache read buffer sizes configurable and improve cache concurrency.
(Mark Miller)
- SOLR-9264: Optimize ZkController.publishAndWaitForDownStates to not read all collection states and
watch relevant collections instead.
(Hrishikesh Gadre, shalin)
- SOLR-9335: Solr cache/search/update stats counters now use LongAdder which are supposed to have higher throughput
under high contention.
(Varun Thacker)
- SOLR-9350: JSON Facets: method="stream" will no longer always uses & populates the filter cache, likely
flushing it. 'cacheDf' can be configured to set a doc frequency threshold, now defaulting to 1/16th doc count.
Using -1 Disables use of the cache.
(David Smiley, yonik)
- Other Changes (23)
- SOLR-9195: Remove unnecessary allocation and null check in UpdateRequestProcessorChain's
getReqProcessors method.
(Christine Poerschke)
- SOLR-8981: Upgraded Extraction module to Apache Tika 1.13.
(Tim Allison, Lewis John McGibbney via Uwe Schindler)
- SOLR-9076: Update to Hadoop 2.7.2
(Mark Miller, Gregory Chanan)
- SOLR-8787: TestAuthenticationFramework should not extend TestMiniSolrCloudCluster.
(Trey Cahill via shalin)
- SOLR-9180: More comprehensive tests of pseudo-fields for RTG and SolrCloud requests
(hossman)
- SOLR-7930: Comment out trappy references to example docs in elevate.xml files
(Erick Erickson)
- SOLR-9277: Clean up some more remnants of supporting old and new style solr.xml in tests
(Erick Erickson)
- SOLR-9163: Sync up basic_configs and data_driven_schema_configs, removing almost all differences
except what is required for schemaless.
(yonik)
- SOLR-9340: Change ZooKeeper disconnect and session expiry related logging from INFO to WARN to
make debugging easier
(Varun Thacker)
- SOLR-9358: [AngularUI] In Cloud->Tree file view area, collapse metadata by default
(janhoy)
- SOLR-9256: asserting hasNext() contract in JdbcDataSource in DataImportHandler
(Kristine Jetzke via Mikhai Khludnev)
- SOLR-9209: extracting JdbcDataSource.createResultSetIterator() for extension
(Kristine Jetzke via Mikhai Khludnev)
- SOLR-9353: Factor out ReRankQParserPlugin.ReRankQueryRescorer private class.
(Christine Poerschke)
- SOLR-9392: Fixed CDCR Test failures which were due to leaked resources.
(shalin)
- SOLR-9385: Add QParser.getParser(String,SolrQueryRequest) variant.
(Christine Poerschke)
- SOLR-9367: Improved TestInjection's randomization logic to use LuceneTestCase.random()
(hossman)
- SOLR-9331: Remove ReRankQuery's length constructor argument and member.
(Christine Poerschke)
- SOLR-9092: For the delete replica command we attempt to send the core admin delete request only
if that node is actually up.
(Jessica Cheng Mallet, Varun Thacker)
- SOLR-9410: Make ReRankQParserPlugin's private ReRankWeight a public class of its own.
(Christine Poerschke)
- SOLR-9404: Refactor move/renames in JSON FacetProcessor and FacetFieldProcessor.
(David Smiley)
- SOLR-9421: Refactored out OverseerCollectionMessageHandler to smaller classes
(noble)
- SOLR-8643: BlockJoinFacetComponent is substituted by BlockJoinFacetDocSetComponent. It doesn't need to change solrconfig.xml
(Mikhail Khludnev)
- SOLR-8644: Test asserts that block join facets work with parent level fq exclusions.
(Dr. Oleg Savrasov via Mikhail Khludnev)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.12.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.8.v20160314
- Upgrading from Solr any prior release (1)
- If you use historical dates, specifically on or before the year 1582, you should re-index.
Detailed Change List
- New Features (33)
- SOLR-8782: Add asynchronous sugar methods to the SolrJ Collections API. You
can now call .processAsync() to run a method asynchronously, or
.processAndWait() to wait for a call to finish without holding HTTP
collections open.
(Alan Woodward)
- SOLR-8765: Enforce required parameters at query construction time in the SolrJ
Collections API, add static factory methods, and deprecate old setter methods.
(Alan Woodward, Jason Gerlowski)
- SOLR-8842: authorization APIs do not use name as an identifier for a permission
for update, delete commands and 'before' attribute
(noble)
- SOLR-8814: Support GeoJSON response writer and general spatial formatting. Adding
&wt=geojson&geojson.field=<your geometry field>
Will return a FeatureCollection for each SolrDocumentList and a Feature with the
requested geometry for each SolrDocument. The requested geometry field needs
to either extend AbstractSpatialFieldType or store a GeoJSON string. This also adds
a [geo] DocumentTransformer that can return the Shape in a variety of formats:
&fl=[geo f=<your geometry field> w=(GeoJSON|WKT|POLY)]
The default format is GeoJSON. For information on the supported formats, see:
https://github.com/locationtech/spatial4j/blob/master/FORMATS.md
To return the FeatureCollection as the root element, add '&omitHeader=true"
(ryan)
- SOLR-8859: Spatial fields like RPT can now be configured to use Spatial4j registered shape formats
e.g. via format="GeoJSON".
(ryan, David Smiley)
- SOLR-445: new TolerantUpdateProcessorFactory to support skipping update commands that cause
failures when sending multiple updates in a single request.
(Erick Erickson, Tomás Fernández Löbbe, Anshum Gupta, hossman)
- SOLR-8890: New static method in DistributedUpdateProcessorFactory to allow UpdateProcessorFactories
to indicate request params that should be forwarded when DUP distributes updates.
(hossman)
- SOLR-8888: Add shortestPath Streaming Expression. This expression performs a breadth first
graph traversal to find the shortest path(s) in an unweighted, directed graph stored in a
SolrCloud collection.
(Joel Bernstein)
- SOLR-8938: Add optional --excluderegex argument to ZkCLI.
(Christine Poerschke)
- SOLR-8976: Add SolrJ support for REBALANCELEADERS Collections API
(Anshum Gupta)
- SOLR-8962: Add sort Streaming Expression. The expression takes a single input stream and a
comparator and outputs tuples in stable order of the comparator.
(Dennis Gove)
- SOLR-8349: Allow sharing of large in memory data structures across cores
(Gus Heck, noble)
- SOLR-9009: Adds ability to get an Explanation of a Streaming Expression
(Dennis Gove)
- SOLR-8918: Adds Streaming to the admin page under the collections section. Includes
ability to see graphically the expression explanation
(Dennis Gove)
- SOLR-8913: When using a shared filesystem we should store data dir and tlog dir locations in
the cluster state.
(Mark Miller)
- SOLR-8809: Implement Connection.prepareStatement
(Kevin Risden)
- SOLR-9020: Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods
(Kevin Risden)
- SOLR-9041: 'core-admin-read' and 'core-admin-edit' are well known permissions
(noble)
- SOLR-8986: Add Random Streaming Expression
(Joel Bernstein)
- SOLR-8925: Add gatherNodes Streaming Expression to support breadth first traversals
(Joel Bernstein)
- SOLR-9027: Add GraphTermsQuery to limit traversal on high frequency nodes
(Joel Bernstein, David Smiley)
- SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE assuming access to a shared file system.
(Varun Thacker, David Smiley)
- SOLR-9049: RuleBasedAuthorizationPlugin supports regex in param values eg: "command" : "REGEX:(i?)create"
(noble)
- SOLR-8972: Add GraphHandler and GraphMLResponseWriter to support graph visualizations
(Joel Bernstein)
- SOLR-9026: Extend facet telemetry support to legacy (non-json) facets under "debug/facet-debug" in
the response.
(Michael Sun, yonik)
- SOLR-7117: Provide an option to limit the maximum number of cores that can be created on a node by the
Auto Add Replica feature. For this you can set a "maxCoresPerNode" property via the Cluster Property API
(Varun Thacker, Mark Miller)
- SOLR-8208: [subquery] document transformer executes separate requests per result document.
(Cao Manh Dat via Mikhail Khludnev)
- SOLR-8323, SOLR-9113: Add CollectionStateWatcher API
(Alan Woodward, Scott Blum)
- SOLR-8988: Adds query option facet.distrib.mco which when set to true allows the use of facet.mincount=1 in cloud mode.
(Keith Laban, Dennis Gove)
- SOLR-8583: Apply highlighting to hl.alternateField by default for Default and FastVectorHighlighter.
Turn off with hl.highlightAlternate=false
(janhoy, David Smiley)
- SOLR-7123: '/update/json/docs' path supports nested documents
(noble)
- SOLR-8610: Resolve variables in encryptKeyFile of DIH's JdbcDataSource
(Kristine Jetzke via Mikhail Khludnev)
- SOLR-7739: Add a new ClassificationUpdateProcessorFactory.
(Alessandro Benedetti via Tommaso Teofili)
- Bug Fixes (18)
- SOLR-8855: The HDFS BlockDirectory should not clean up it's cache on shutdown.
(Mark Miller)
- SOLR-8948: OverseerTaskQueue.containsTaskWithRequestId encounters json parse error if a
SolrResponse node is in the overseer queue.
(Jessica Cheng Mallet via shalin)
- SOLR-7729: ConcurrentUpdateSolrClient ignores the collection parameter in some methods.
(Nicolas Gavalda, Jorge Luis Betancourt Gonzalez via Mark Miller)
- SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new change.
(sarowe, Noble Paul, Varun Thacker)
- SOLR-8983: Cleanup clusterstate and replicas for a failed create collection request
(Varun Thacker, Anshum Gupta)
- SOLR-9029: fix rare ZkStateReader visibility race during collection state format update
(Scott Blum, hossman)
- SOLR-9046: Fix solr.cmd that wrongly assumes Jetty will always listen on 0.0.0.0.
(Bram Van Dam, Uwe Schindler)
- SOLR-9064: Adds an explanation of the incoming stream to an UpdateStream's explanation
(Dennis Gove)
- SOLR-9128: Fix error handling issues in Streaming classes
(Joel Bernstein)
- SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD
(janhoy)
- SOLR-9141: Fix ClassCastException when using the /sql handler count() function with
single-shard collections
(Minoru Osuka via James Dyer)
- SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used with "cursorMark".
(James Dyer)
- SOLR-8940: Fix group.sort option
(hossman)
- SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH)
(Kristine Jetzke via Mikhail Khludnev)
- SOLR-8676: keep LOG4J_CONFIG in solr.cmd
(Kristine Jetzke via Mikhail Khludnev)
- SOLR-9198: config APIs unable to add multiple values with same name
(noble)
- SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided
and there are explicit operators (except for AND) - addresses problems caused by SOLR-2649.
(Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe)
- SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to
FCS, causing slowdowns
(Alessandro Benedetti, Jesse McLaughlin, Alan Woodward)
- Optimizations (11)
- SOLR-8722: Don't force a full ZkStateReader refresh on every Overseer operation.
(Scott Blum via shalin)
- SOLR-8745: Deprecate costly ZkStateReader.updateClusterState(), replace with a narrow
forceUpdateCollection(collection)
(Scott Blum via shalin)
- SOLR-8856: Do not cache merge or 'read once' contexts in the hdfs block cache.
(Mark Miller, Mike Drob)
- SOLR-8922: Optimize filter creation (DocSetCollector) to minimize the amount of garbage
produced. This resulted in up to 3x throughput when small filter creation was the bottleneck,
as well as orders of magnitude less garbage.
(Jeff Wartes, yonik)
- SOLR-8937: bin/post (SimplePostTool) now streams the standard input instead of buffering fully.
(David Smiley)
- SOLR-8973: Zookeeper frenzy when a core is first created.
(Janmejay Singh, Scott Blum, shalin)
- SOLR-9014: Deprecate and reduce usage of ClusterState methods which may make calls to ZK via
the lazy collection reference.
(Scott Blum, shalin)
- SOLR-9106: Cluster properties are now cached on ZkStateReader.
(Alan Woodward)
- SOLR-9147: Upgrade commons-io to 2.5, avoid expensive array resizing in EmbeddedSolrServer
(Mikhail Khludnev)
- SOLR-8744: Overseer operations performed with fine grained mutual exclusion
(noble, Scott Blum)
- SOLR-9204: Improve performance of getting directory size with hdfs.
(Mark Miller)
- Other Changes (27)
- SOLR-8860: Remove back-compat handling of router format made in SOLR-4221 in 4.5.0.
(shalin)
- SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to serialize a value.
(David Smiley)
- SOLR-8842: security rules made more foolproof by asking the requesthandler about the well known
permission name. The APIs are also modified to ue 'index' as the unique identifier instead of name.
Name is an optional attribute now and only to be used when specifying well-known permissions
(noble)
- SOLR-5616: Simplifies grouping code to use ResponseBuilder.needDocList() to determine if it needs to
generate a doc list for grouped results.
(Steven Bower, Keith Laban, Dennis Gove)
- SOLR-8869: Optionally disable printing field cache entries in SolrFieldCacheMBean
(Gregory Chanan)
- SOLR-8892: Allow SolrInfoMBeans to return different statistics for /jmx vs web ui calls.
(Gregory Chanan, Mark Miller)
- SOLR-8097: Implement builder pattern design for constructing SolrJ clients and also deprecate direct construction
of client objects.
(Jason Gerlowski, Shawn Heisey, Anshum Gupta)
- SOLR-9015: Adds SelectStream as a default function in the StreamHandler
(Dennis Gove)
- SOLR-8929: Add an idea module for solr/server to enable launching start.jar
(Scott Blum, Steve Rowe)
- SOLR-8933: Solr should not close container streams.
(Mike Drob, Uwe Schindler, Mark Miller)
- SOLR-9037: Replace multiple "/replication" strings with one static constant.
(Christine Poerschke)
- SOLR-9047: zkcli should allow alternative locations for log4j configuration
(Gregory Chanan)
- SOLR-9066: Make CountMetric return long instead of double
(Kevin Risden)
- SOLR-9065, SOLR-9072, SOLR-9132: Migrate some distributed tests to SolrCloudTestCase.
(Alan Woodward)
- SOLR-8184: Negative tests for JDBC Connection String
(Susheel Kumar, Jason Gerlowski, Kevin Risden)
- SOLR-8458: Add Streaming Expressions tests for parameter substitution
(Joel Bernstein, Cao Manh Dat, Dennis Gove, Kevin Risden)
- SOLR-8467: CloudSolrStream and FacetStream should take a SolrParams object rather than a
Map<String, String> to allow more complex Solr queries to be specified.
(Erick Erickson)
- SOLR-9083: Remove all <types> and <fields> from schemas. NOTE: as in the JIRA I left a few in to insure the (no cost)
back compat.
(Erick Erickson)
- SOLR-9105: Fix a bunch of typos across 103 files
(Bartosz Krasiński via janhoy)
- SOLR-9159: New cloud based concurrent atomic update test
(hossman)
- SOLR-9119: several static methods in ValueSourceParser have been made private
(hossman)
- SOLR-9110: Move JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase
(Mikhail Khludnev)
- SOLR-9136: Separate out the error statistics into server-side error vs client-side error
(Jessica Cheng Mallet via Erick Erickson)
- SOLR-9107: new @RandomizeSSL annotation for more fine grained control of SSL testing
(hossman, sarowe)
- SOLR-9081: Make SolrTestCaseJ4.beforeClass() / .afterClass() public so it
works with Mockito
(Georg Sorst, Alan Woodward)
- SOLR-8445: fix line separator in log4j.properties files
(Ahmet Arslan via Mikhail Khludnev)
- SOLR-2199: DataImportHandler (DIH) JdbcDataSource supports multiple resultsets per query
(Kristine Jetzke, Mark Waddle via Mikhail Khludnev)
- Upgrade Notes (1)
- If you use historical dates, specifically on or before the year 1582, you should re-index.
- Bug Fixes (24)
- SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe.
(Scott Blum, hoss,
sarowe, Erick Erickson, Mark Miller, shalin)
- SOLR-9016: Fix SolrIdentifierValidator to not allow empty identifiers.
(Shai Erera)
- SOLR-8992: Restore Schema API GET method functionality removed in 6.0
(noble, Steve Rowe)
- SOLR-9080, SOLR-9085: (6.0 bug) For years <= 1582, date math (round,add,sub) introduced error. Range faceting
on such dates was also affected. With this fixed, this is the first release range faceting works on BC years.
(David Smiley)
- SOLR-8857: HdfsUpdateLog does not use configured or new default number of version buckets and is
hard coded to 256.
(Mark Miller, yonik, Gregory Chanan)
- SOLR-8902: Make sure ReturnFields only returns the requested fields from (fl=) evn when
DocumentTransformers ask for getExtraRequestFields()
(ryan)
- SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting in NPE.
(Scott Blum via David Smiley)
- SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes.
(David Smiley)
- SOLR-9004: Fix "name" field type definition in films example.
(Alexandre Rafalovitch via Varun Thacker)
- SOLR-8990: Fix top term links from schema browser page to use {!term} parser
(hossman)
- SOLR-8971: Preserve root cause when wrapping exceptions
(hossman)
- SOLR-9034: Atomic updates failed to work when there were copyField targets that had docValues
enabled.
(Karthik Ramachandran, Ishan Chattopadhyaya, yonik)
- SOLR-9028: Fixed some test related bugs preventing SSL + ClientAuth from ever being tested
(hossman)
- SOLR-9059: NPE in SolrClientCache following collection reload
(Joel Bernstein, Ryan Yacyshyn)
- SOLR-8792: ZooKeeper ACL support fixed.
(Esther Quansah, Ishan Chattopadhyaya, Steve Rowe)
- SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter.
(Scott Blum, Mark Miller, shalin)
- SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart.
(Lior Sapir, Mark Miller, shalin)
- SOLR-9058: Makes HashJoinStream and OuterHashJoinStream support different field names in the
incoming streams, eg. fieldA=fieldB.
(Dennis Gove, Stephan Osthold)
- SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor.
(Christine Poerschke)
- SOLR-9118: HashQParserPlugin should trim partition keys
(Joel Bernstein)
- SOLR-9117: The first SolrCore is leaked after reload.
(Jessica Cheng Mallet via shalin)
- SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded.
(Jessica Cheng Mallet via shalin)
- SOLR-8801: /bin/solr create script always returns exit code 0 when a collection/core already exists.
(Khalid Alharbi, Marius Grama via Steve Rowe)
- SOLR-9134: Fix RestManager.addManagedResource return value.
(Christine Poerschke)
- Other Changes (7)
- SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the single-usage policy.
(Jason Gerlowski, Benoit Vanalderweireldt, shalin)
- SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed
anymore. The Replication tab is only beneficial to users running Solr in master-slave mode.
(Varun Thacker)
- SOLR-8985: Added back support for 'includeDynamic' flag to /schema/fields endpoint
(noble)
- SOLR-9131: Fix "start solr" text in cluster.vm Velocity template
(janhoy)
- SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability
(Jeff Field, Mike Drob via janhoy)
- SOLR-9115: Get rid of javax.xml.bind.DatatypeConverter in SimplePostTool
for Java 9 compatibility.
(Uwe Schindler)
- SOLR-5776,SOLR-9068,SOLR-8970:
-
Refactor SSLConfig so that SSLTestConfig can provide SSLContexts using a NullSecureRandom
to prevent SSL tests from blocking on entropy starved machines.
-
SSLTestConfig: Alternate (pseudo random) NullSecureRandom for Constants.SUN_OS.
-
SSLTestConfig: Replace NullSecureRandom w/ NotSecurePseudoRandom.
-
Change SSLTestConfig to use a keystore file that is included as a resource in the
test-framework jar so users subclassing SolrTestCaseJ4 don't need to preserve magic paths.
(hossman)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.12.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.8.v20160314
- System Requirements (1)
- LUCENE-5950: Move to Java 8 as minimum Java version.
(Ryan Ernst, Uwe Schindler)
- Upgrading from Solr 5.x (18)
- The deprecated SolrServer and subclasses have been removed, use SolrClient
instead.
- The deprecated <nrtMode> configuration in solrconfig.xml has been removed.
Please remove it from solrconfig.xml.
- SolrClient.shutdown() has been removed, use SolrClient.close() instead.
- The deprecated zkCredientialsProvider element in solrcloud section of solr.xml
is now removed. Use the correct spelling (zkCredentialsProvider) instead.
- SOLR-7957: internal/expert - ResultContext was significantly changed and expanded
to allow for multiple full query results (DocLists) per Solr request.
TransformContext was rendered redundant and was removed.
(yonik)
- Several changes have been made regarding the "Similarity" used in Solr, in order to provide
better default behavior for new users. There are 3 key impacts of these changes on existing
users who upgrade:
- DefaultSimilarityFactory has been removed. If you currently have DefaultSimilarityFactory explicitly
referenced in your schema.xml, edit your config to use the functionally identical
ClassicSimilarityFactory. See SOLR-8239 for more details.
- The implicit default Similarity used when no <similarity/> is configured in schema.xml has
been changed to SchemaSimilarityFactory. Users who wish to preserve back-compatible behavior should
either explicitly configure ClassicSimilarityFactory, or ensure that the luceneMatchVersion
for the collection is less then 6.0. See SOLR-8270 + SOLR-8271 for details.
- SchemaSimilarityFactory has been modified to use BM25Similarity as the default for fieldTypes that
do not explicitly declare a Similarity. The legacy behavior of using ClassicSimilarity as the
default will occur if the luceneMatchVersion for the collection is less then 6.0, or the
'defaultSimFromFieldType' configuration option may be used to specify any default of your choosing.
See SOLR-8261 + SOLR-8329 for more details.
- If your solrconfig.xml file doesn't explicitly mention the schemaFactory to use then Solr will choose
the ManagedIndexSchemaFactory by default. Previously it would have chosen ClassicIndexSchemaFactory.
This means that the Schema APIs ( /<collection>/schema ) are enabled and the schema is mutable.
When Solr starts your schema.xml file will be renamed to managed-schema. If you want to retain the old behaviour
then please ensure that the solrconfig.xml explicitly uses the ClassicIndexSchemaFactory :
<schemaFactory class="ClassicIndexSchemaFactory"/> or your luceneMatchVersion in the solrconfig.xml is less than 6.0
- SolrIndexSearcher.QueryCommand and QueryResult were moved to their own classes. If you reference them
in your code, you should import them under o.a.s.search (or use your IDE's "Organize Imports").
- SOLR-8698: 'useParams' attribute specified in request handler cannot be overridden from request params
- When requesting stats in date fields, "sum" is now a double value instead of a date. See SOLR-8671
- SOLR-8736: The deprecated GET methods for schema are now accessible through the bulk API. The output
has less details and is not backward compatible.
- In the past, Solr guaranteed that retrieval of multi-valued fields would preserve the order of values.
Because values may now be retrieved from column-stored fields (docValues="true"), in conjunction with the
fact that docValues do not currently preserve order, means that users should set useDocValuesAsStored="false"
to prevent future optizations from using the column-stored values over the row-stored values when
fields have both stored="true" and docValues="true".
- Formatted date-times from Solr have some differences. If the year is more than 4 digits, there is a leading '+'.
When there is a non-zero number of milliseconds, it is padded with zeros to 3 digits. Negative year (BC) dates are
now possible. Parsing: It is now an error to supply a portion of the date out of its, range, like 67 seconds.
- SolrJ no longer includes DateUtil. If for some reason you need to format or parse dates, simply use Instant.format()
and Instant.parse().
- If you are using an RPT or other spatial field referencing Spatial4j in its configuration, then replace the string
"com.spatial4j.core" with "org.locationtech.spatial4j". Consider updating direct to Solr 6.2 which accepts the
old value, albeit with a warning.
Detailed Change List
- New Features (33)
- SOLR-3085: New edismax param mm.autoRelax which helps in certain cases of the stopwords/zero-hits issue
(janhoy)
- SOLR-7560: Parallel SQL Support
(Joel Bernstein)
- SOLR-7707: Add StreamExpression Support to RollupStream
(Dennis Gove, Joel Bernstein)
- SOLR-7903: Add the FacetStream to the Streaming API and wire it into the SQLHandler
(Joel Bernstein)
- SOLR-7986: JDBC Driver for SQL Interface
(Uwe Schindler, Joel Bernstein)
- SOLR-8038: Add the StatsStream to the Streaming API and wire it into the SQLHandler
(Joel Bernstein)
- SOLR-8086: Add support for SELECT DISTINCT queries to the SQL interface
(Joel Bernstein)
- SOLR-7543: Basic graph traversal query
Example: {!graph from="node_id" to="edge_id"}id:doc_1
(Kevin Watters, yonik)
- SOLR-6273: Cross Data Center Replication. Active/passive replication for separate
SolrClouds hosted on separate data centers.
(Renaud Delbru, Yonik Seeley via Erick Erickson)
- SOLR-7938: MergeStream now supports merging more than 2 streams together
(Dennis Gove)
- SOLR-8198: Change ReducerStream to use StreamEqualitor instead of StreamComparator
(Dennis Gove)
- SOLR-8268: StatsStream now implements the Expressible interface
(Dennis Gove)
- SOLR-7584: Adds Inner and LeftOuter Joins to the Streaming API and Streaming Expressions
(Dennis Gove, Corey Wu)
- SOLR-8188: Adds Hash and OuterHash Joins to the Streaming API and Streaming Expressions
(Dennis Gove)
- SOLR-7669: Add SelectStream and Tuple Operations to the Streaming API and Streaming Expressions
(Dennis Gove)
- SOLR-8337: Add ReduceOperation and wire it into the ReducerStream
(Joel Bernstein)
- SOLR-7904: Add StreamExpression Support to FacetStream
(Dennis Gove)
- SOLR-6398: Add IterativeMergeStrategy to support running Parallel Iterative Algorithms inside of Solr
(Joel Bernstein)
- SOLR-8436: Real-time get now supports filters.
(yonik)
- SOLR-7535: Add UpdateStream to Streaming API and Streaming Expression
(Jason Gerlowski, Joel Bernstein)
- SOLR-8479: Add JDBCStream to Streaming API and Streaming Expressions for integration with external data sources
(Dennis Gove)
- SOLR-8002: Add column alias support to the Parallel SQL Interface
(Joel Bernstein)
- SOLR-7525: Add ComplementStream and IntersectStream to the Streaming API and Streaming Expressions
(Dennis Gove, Jason Gerlowski, Joel Bernstein)
- SOLR-8415: Provide command to switch between non/secure mode in ZK
(Mike Drob, Gregory Chanan)
- SOLR-8556: Add ConcatOperation to be used with the SelectStream
(Joel Bernstein, Dennis Gove)
- SOLR-8550: Add asynchronous DaemonStreams to the Streaming API
(Joel Bernstein)
- SOLR-8285: Ensure the /export handler works with NULL field values
(Joel Bernstein)
- SOLR-8502: Improve Solr JDBC Driver to support SQL Clients like DBVisualizer
(Kevin Risden, Joel Bernstein)
- SOLR-8588: Add TopicStream to the streaming API to support publish/subscribe messaging
(Joel Bernstein, Kevin Risden)
- SOLR-8666: Adds header 'zkConnected' to response of SearchHandler and PingRequestHandler to notify the client when
a connection to zookeeper has been lost and there is a possibility of stale data on the node the request is coming
from.
(Keith Laban, Dennis Gove)
- SOLR-8522: Make it possible to use ip fragments in replica placement rules , such as ip_1, ip_2 etc
(Arcadius Ahouansou, noble)
- SOLR-8698: params.json can now specify 'appends' and 'invariants'
(noble)
- SOLR-8831: allow _version_ field to be retrievable via docValues
(yonik)
- Bug Fixes (39)
- SOLR-8386: Add field option in the new admin UI schema page loads up even when no schemaFactory has been
explicitly specified since the default is ManagedIndexSchemaFactory.
(Erick Erickson, Upayavira, Varun Thacker)
- SOLR-8191: Guard against CloudSolrStream close method NullPointerException
(Kevin Risden, Joel Bernstein)
- SOLR-8485: SelectStream now properly handles non-lowercase and/or quoted select field names
(Dennis Gove)
- SOLR-8525: Fix a few places that were failing to pass dimensional
values settings when copying a FieldInfo
(Ishan Chattopadhyaya via
Mike McCandless)
- SOLR-8409: Ensures that quotes in solr params (eg. q param) are properly handled
(Dennis Gove)
- SOLR-8640: CloudSolrClient does not send credentials for update request
(noble, hoss)
- SOLR-8461: CloudSolrStream and ParallelStream can choose replicas that are not active
(Cao Manh Dat, Varun Thacker, Joel Bernstein)
- SOLR-8527: Improve JdbcTest to cleanup properly on failures
(Kevin Risden, Joel Bernstein)
- SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we
count on HttpClient or the JVM.
(Mark Miller)
- SOLR-8683: Always consume the full request on the server, not just in the case of an error.
(Mark Miller)
- SOLR-8416: The collections create API should return after all replicas are active.
(Michael Sun, Mark Miller, Alexey Serba)
- SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a request too early.
(Mark Miller)
- SOLR-8694: DistributedMap/Queue can create too many Watchers and some code simplification.
(Scott Blum via Mark Miller)
- SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection events and
make this handling more consistent.
(Scott Blum via Mark Miller)
- SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and
SolrCmdDistributor, which violates the lifecycle and can cause bugs.
(hossman via Mark Miller)
- SOLR-8656: PeerSync should use same nUpdates everywhere.
(Ramsey Haddad via Mark Miller)
- SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections from interfering with each other
and other small LeaderElector improvements.
(Scott Blum via Mark Miller, Alan
Woodward)
- SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection
can be made.
(Keith Laban, Dennis Gove)
- SOLR-8497: Merge index does not mark the Directory objects it creates as 'done' and they are retained in the
Directory cache.
(Sivlio Sanchez, Mark Miller)
- SOLR-8696: Start the Overseer before actions that need the overseer on init and when reconnecting after
zk expiration and improve init logic.
(Scott Blum, Mark Miller)
- SOLR-8420: Fix long overflow in sumOfSquares for Date statistics.
(Tom Hill, Christine Poerschke,
Tomás Fernández Löbbe)
- SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10 even though the thread pool
size is 100. The limit has now been increased to 100.
(Scott Blum, shalin)
- SOLR-8375: ReplicaAssigner rejects valid nodes
(Kelvin Tan, noble)
- SOLR-8738: Fixed false success response when invalid deleteByQuery requests intially hit non-leader
cloud nodes
(hossman)
- SOLR-8771: Multi-threaded core shutdown creates executor per core.
(Mike Drob via Mark Miller)
- SOLR-8145: Fix position of OOM killer script when starting Solr in the background
(Jurian Broertjes via
Timothy Potter)
- SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas that have non-"id"
unique field
(Erik Hatcher, Anshum Gupta)
- SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica
placement. splitshard should preassign nodes using rules, if rules are present
(noble, Shai Erera)
- SOLR-8779: Fix missing InterruptedException handling in ZkStateReader.java
(Varun Thacker)
- SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core
(Johannes Brucher, Varun Thacker)
- SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without
docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort
other than count, resulted in incorrect results. This has been fixed, and facet.prefix
support for facet.method=uif has been enabled.
(Mikhail Khludnev, yonik)
- SOLR-8790: Collections API responses contain node name in the core-level responses that are
returned.
(Anshum Gupta)
- SOLR-8804: Fix a race condition in the ClusterStatus API call whereby the call would fail when a concurrent delete
collection api command was executed
(Alexey Serba, Varun Thacker)
- SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields that
have docValues.
(yonik)
- SOLR-8838: Returning non-stored docValues is incorrect for negative floats and doubles.
(Ishan Chattopadhyaya, Steve Rowe)
- SOLR-8867: {!frange} queries will now avoid matching documents without a value in the
numeric field. For more complex functions, FunctionValues.exists() must also return true
for the document to match.
(yonik)
- SOLR-8886: Fix TrieField.toObject(IndexableField) to work for field with docValues
enabled.
(yonik)
- SOLR-8891: Fix StrField.toObject and toExternal to work with docValue IndexableField
instances.
(yonik)
- SOLR-8865: Real-time get sometimes fails to retrieve stored fields from docValues.
(Ishan Chattopadhyaya, yonik)
- Optimizations (7)
- SOLR-7876: Speed up queries and operations that use many terms when timeAllowed has not been
specified. Speedups of up to 8% were observed.
(yonik)
- SOLR-8037: Speed up creation of filters from term range queries (i.e. non-numeric range queries)
and use the filter cache for term range queries that are part of larger queries. Some observed
speedups were up to 2.5x for production of filters, and up to 10x for query evaluation with
embedded term range queres that resulted in filter cache hits.
(yonik)
- SOLR-8559: FCS facet performance optimization which significantly speeds up processing when terms
are high cardinality and the matching docset is small. When facet minCount > 0 and the number of
matching documents is small (or 0) this enhancement prevents considering terms which have a 0
count. Also includes change to move to the next non-zero term value when selecting a segment
position.
(Keith Laban, Steve Bower, Dennis Gove)
- SOLR-8532: Optimize GraphQuery when maxDepth is set by not collecting edges at the maxDepth level.
(Kevin Watters via yonik)
- SOLR-8669: Non binary responses use chunked encoding because we flush the outputstream early.
(Mark Miller)
- SOLR-8720: ZkController#publishAndWaitForDownStates should use #publishNodeAsDown.
(Mark Miller)
- SOLR-8082: Can't query against negative float or double values when indexed="false"
docValues="true" multiValued="false".
(hossman, Ishan Chattopadhyaya, yonik, Steve Rowe)
- Other Changes (46)
- SOLR-6127: Improve example docs, using films data
(Varun Thacker via ehatcher)
- SOLR-6895: Deprecated SolrServer classes have been removed
(Alan Woodward,
Erik Hatcher)
- SOLR-6954: Deprecated SolrClient.shutdown() method removed
(Alan Woodward)
- SOLR-7355: Switch from Google's ConcurrentLinkedHashMap to Caffeine. Only
affects HDFS support.
(Ben Manes via Shawn Heisey)
- SOLR-7624: Remove deprecated zkCredientialsProvider element in solrcloud section of solr.xml.
(Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller)
- SOLR-7513: Add Equalitors to Streaming Expressions
(Dennis Gove, Joel Bernstein)
- SOLR-7528: Simplify Interfaces used in Streaming Expressions
(Dennis Gove, Joel Bernstein)
- SOLR-7554: Add checks in Streams for incoming stream order
(Dennis Gove, Joel Bernstein)
- SOLR-7441: Improve overall robustness of the Streaming stack: Streaming API,
Streaming Expressions, Parallel SQL
(Joel Bernstein)
- SOLR-8153: Support upper case and mixed case column identifiers in the SQL interface
(Joel Bernstein)
- SOLR-8132: HDFSDirectoryFactory now defaults to using the global block cache.
(Mark Miller)
- SOLR-8261: Change SchemaSimilarityFactory default to BM25Similarity
(hossman)
- SOLR-8259: Remove deprecated JettySolrRunner.getDispatchFilter()
- SOLR-8258: Change default hdfs tlog replication factor from 1 to 3.
(Mark Miller)
- SOLR-8270: Change implicit default Similarity to use BM25 when luceneMatchVersion >= 6
(hossman)
- SOLR-8271: Change implicit default Similarity to use SchemaSimilarityFactory when luceneMatchVersion >= 6
(hossman)
- SOLR-8179: SQL JDBC - DriverImpl loadParams doesn't support keys with no values in the connection string
(Kevin Risden, Joel Bernstein)
- SOLR-8131: Make ManagedIndexSchemaFactory the default schemaFactory when luceneMatchVersion >= 6
(Uwe Schindler, shalin, Varun Thacker)
- SOLR-8266: Remove Java Serialization from the Streaming API. The /stream handler now only accepts
Streaming Expressions.
(Jason Gerlowski, Joel Bernstein)
- SOLR-8426: Enable /export, /stream and /sql handlers by default and remove them from example configs.
(shalin)
- SOLR-8443: Change /stream handler http param from "stream" to "expr"
(Joel Bernstein, Dennis Gove)
- SOLR-5209: Unloading or deleting the last replica of a shard now no longer
cascades to remove the shard from the clusterstate.
(Christine Poerschke)
- SOLR-8190: Implement Closeable on TupleStream
(Kevin Risden, Joel Bernstein)
- SOLR-8529: Improve JdbcTest to not use plain assert statements
(Kevin Risden, Joel Bernstein)
- SOLR-7339: Upgrade Jetty to v9.3.8.v20160314.
(Gregg Donovan, shalin, Mark Miller, Steve Rowe)
- SOLR-5730: Make Lucene's SortingMergePolicy and EarlyTerminatingSortingCollector configurable in Solr.
(Christine Poerschke, hossmann, Tomás Fernández Löbbe, Shai Erera)
- SOLR-8677: Prevent shards containing invalid characters from being created. Checks added server-side
and in SolrJ.
(Shai Erera, Jason Gerlowski, Anshum Gupta)
- SOLR-8693: Improve ZkStateReader logging.
(Scott Blum via Mark Miller)
- SOLR-8710: Upgrade morfologik-stemming to version 2.1.0.
(Dawid Weiss)
- SOLR-8711: Upgrade Carrot2 clustering dependency to 3.12.0.
(Dawid Weiss)
- SOLR-8690: Make peersync fingerprinting optional with solr.disableFingerprint system
property.
(yonik)
- SOLR-8691: Cache index fingerprints per searcher.
(yonik)
- SOLR-8746: Renamed Overseer.getInQueue to getStateUpdateQueue, getInternalQueue to getInternalWorkQueue
and added javadocs.
(Scott Blum, shalin)
- SOLR-8752: Add a test for SizeLimitedDistributedMap and improve javadocs.
(shalin)
- SOLR-8671: Date statistics: make "sum" a double instead of a long/date
(Tom Hill, Christine Poerschke,
Tomás Fernández Löbbe)
- SOLR-8713: new UI and example solrconfig files point to Reference Guide for Solr Query Syntax instead
of the wiki.
(Marius Grama via Tomás Fernández Löbbe)
- SOLR-8758: Add a new SolrCloudTestCase class, using MiniSolrCloudCluster
(Alan
Woodward)
- SOLR-8736: schema GET operations on fields, dynamicFields, fieldTypes, copyField are
reimplemented as a part of the bulk API with less details
(noble)
- SOLR-8766: Remove deprecated <admin> tag in solrconfig.xml and support for admin/gettableFiles
(noble, Jason Gerlowski, Varun Thacker)
- SOLR-8799: Improve error message when tuple can't be read by SolrJ JDBC
(Kevin Risden, Joel Bernstein)
- SOLR-8836: Return 400, and a SolrException when an invalid json is provided to the update handler
instead of 500.
(Jason Gerlowski via Anshum Gupta)
- SOLR-8740: docValues are now enabled by default for most non-text (string, date, and numeric) fields
in the schema templates.
(yonik)
- SOLR-8819: Implement DatabaseMetaDataImpl getTables() and fix getSchemas().
(Trey Cahill,
Joel Bernstein, Kevin Risden)
- SOLR-8810: Implement Connection.setReadOnly, Statement.set/getFetchSize,
ResultSet.getType
(Kevin Risden)
- SOLR-8904: All dates are formatted via Instant.toString() (ISO-8601); see Solr upgrade notes for differences. Will
now parse (and format) dates with a leading '+' or '-' (BC dates or dates > 4 digit year.
[value] and ms() and contrib/analytics now parse with date math.
(David Smiley)
- SOLR-8904: DateUtil in SolrJ moved to the extraction contrib as ExtractionDateUtil. Obsolete methods were removed.
(David Smiley)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.10.4
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.2.13.v20150730
- Upgrade Notes (2)
- SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... })
the resolving of external entities is now disallowed by default.
- SOLR-11482: RunExecutableListener was deprecated and is disabled by default for
security reasons. Legacy applications still using it must explicitely pass
'-Dsolr.enableRunExecutableListener=true' to the Solr command line.
Be aware that you should really disable API-based config editing at the same
time, using '-Ddisable.configEdit=true'!
(Uwe Schindler)
- Bug Fixes (2)
- SOLR-10420: Leaking one SolrZkClient instance per second
(Scott Blum, Cao Manh Dat, Markus Jelsma, Steve Rowe)
- SOLR-11477: Disallow resolving of external entities in the XML query parser (defType=xmlparser).
(Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke)
- Other (1)
- SOLR-11486: Update jmatio to 1.2 and prevent deserialization inside Matlab files.
(Uwe Schindler, Steve Rowe)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.13
- Carrot2 3.15.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.14.v20161028
- Other Changes (2)
- SOLR-9819: Upgrade commons-fileupload to 1.3.2, fixing a potential vulnerability CVE-2016-3092
(Anshum Gupta)
- SOLR-10031: Validation of filename params in ReplicationHandler
(Hrishikesh Gadre, janhoy)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (7)
- Apache Tika 1.13
- Carrot2 3.12.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.3.8.v20160314
- (No Changes)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.10.4
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.2.13.v20150730
- Bug Fixes (26)
- SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection events and
make this handling more consistent.
(Scott Blum via Mark Miller)
- SOLR-9198: config APIs unable to add multiple values with same name
(noble)
- SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed
(Scott Blum, Noble Paul)
- SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided
and there are explicit operators (except for AND) - addresses problems caused by SOLR-2649.
(Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe)
- SOLR-9034: Atomic updates failed to work when there were copyField targets that had docValues
enabled.
(Karthik Ramachandran, Ishan Chattopadhyaya, yonik)
- SOLR-8940: Fix group.sort option
(hossman)
- SOLR-8857: HdfsUpdateLog does not use configured or new default number of version buckets and is
hard coded to 256.
(Mark Miller, yonik, Gregory Chanan)
- SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting in NPE.
(Scott Blum via David Smiley)
- SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes.
(David Smiley)
- SOLR-9004: Fix "name" field type definition in films example.
(Alexandre Rafalovitch via Varun Thacker)
- SOLR-8990: Fix top term links from schema browser page to use {!term} parser
(hossman)
- SOLR-8971: Preserve root cause when wrapping exceptions
(hossman)
- SOLR-8792: ZooKeeper ACL support fixed.
(Esther Quansah, Ishan Chattopadhyaya, Steve Rowe)
- SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter.
(Scott Blum, Mark Miller, shalin)
- SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart.
(Lior Sapir, Mark Miller, shalin)
- SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor.
(Christine Poerschke)
- SOLR-9118: HashQParserPlugin should trim partition keys
(Joel Bernstein)
- SOLR-9117: The first SolrCore is leaked after reload.
(Jessica Cheng Mallet via shalin)
- SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded.
(Jessica Cheng Mallet via shalin)
- SOLR-8801: /bin/solr create script always returns exit code 0 when a collection/core already exists.
(Khalid Alharbi, Marius Grama via Steve Rowe)
- SOLR-9134: Fix RestManager.addManagedResource return value.
(Christine Poerschke)
- SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD
(janhoy)
- SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used with "cursorMark".
(James Dyer)
- SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH)
(Kristine Jetzke via Mikhail Khludnev)
- SOLR-8676: keep LOG4J_CONFIG in solr.cmd
(Kristine Jetzke via Mikhail Khludnev)
- SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to
FCS, causing slowdowns
(Alessandro Benedetti, Jesse McLaughlin, Alan Woodward)
- Other Changes (11)
- SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the single-usage policy.
(Jason Gerlowski, Benoit Vanalderweireldt, shalin)
- SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed
anymore. The Replication tab is only beneficial to users running Solr in master-slave mode.
(Varun Thacker)
- SOLR-9131: Fix "start solr" text in cluster.vm Velocity template
(janhoy)
- SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability
(Jeff Field, Mike Drob via janhoy)
- SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to serialize a value.
(David Smiley)
- SOLR-8933: Solr should not close container streams.
(Mike Drob, Uwe Schindler, Mark Miller)
- SOLR-9037: Replace multiple "/replication" strings with one static constant.
(Christine Poerschke)
- SOLR-9047: zkcli should allow alternative locations for log4j configuration
(Gregory Chanan)
- SOLR-9105: Fix a bunch of typos across 103 files
(Bartosz Krasiński via janhoy)
- SOLR-8445: fix line separator in log4j.properties files
(Ahmet Arslan via Mikhail Khludnev)
- SOLR-8674: Stop ignoring sysprop solr.tests.mergePolicy, and make tests randomly choose between
setting <mergePolicy> and <mergePolicyFactory>, which was added in SOLR-8621.
(Christine Poerschke)
- Bug Fixes (42)
- SOLR-8737: Managed synonym lists do not include the original term in the expand
(janhoy)
- SOLR-8734: fix (maxMergeDocs|mergeFactor) deprecation warnings: in solrconfig.xml
<maxMergeDocs|mergeFactor> may not be combined with <mergePolicyFactory> and
<maxMergeDocs|mergeFactor> on their own or combined with <mergePolicy> is a warning.
(Christine Poerschke, Shai Erera)
- SOLR-8712: Variable solr.core.instanceDir was not being resolved
(Kristine
Jetzke, Shawn Heisey, Alan Woodward)
- SOLR-8793: Fix Core admin status API to not fail when computing the size of the segments_N
file if the file no longer exists (for example, if a commit happened and the IndexReader
hasn't refreshed yet). In this case the reported size of the file is -1.
(Shai Erera, Alexey Serba, Richard Coggins)
- SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica
placement. splitshard should preassign nodes using rules, if rules are present
(noble, Shai Erera)
- SOLR-8838: Returning non-stored docValues is incorrect for negative floats and doubles.
(Ishan Chattopadhyaya, Steve Rowe)
- SOLR-8870: AngularJS Query tab no longer URL-encodes the /select part of the request, fixing possible 404 issue
when Solr is behind a proxy. Also, now supports old-style &qt param when handler not prefixed with "/"
(janhoy)
- SOLR-8725: Allow hyphen in collection, core, shard, and alias name as the non-first character
(Anshum Gupta)
(from 6.0)
- SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without
docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort
other than count, resulted in incorrect results. This has been fixed, and facet.prefix
support for facet.method=uif has been enabled.
(Mikhail Khludnev, yonik)
- SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields that
have docValues.
(yonik)
- SOLR-8908: Fix to OnReconnect listener registration to allow listeners to deregister, such
as when a core is reloaded or deleted to avoid a memory leak.
(Timothy Potter)
- SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new change
(sarowe, Noble Paul, Varun Thacker)
- SOLR-9007: Remove mention of the managed_schema_configs as valid config directory when creating
the collection for the SolrCloud example.
(Timothy Potter)
- SOLR-8983: Cleanup clusterstate and replicas for a failed create collection request
(Varun Thacker, Anshum Gupta)
- SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we
count on HttpClient or the JVM.
(Mark Miller)
- SOLR-8683: Always consume the full request on the server, not just in the case of an error.
(Mark Miller)
- SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a request too early.
(Mark Miller)
- SOLR-8416: The collections create API should return after all replicas are active.
(Michael Sun, Mark Miller, Alexey Serba)
- SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe.
(Scott Blum, hoss,
sarowe, Erick Erickson, Mark Miller, shalin)
- SOLR-8973: Zookeeper frenzy when a core is first created.
(Janmejay Singh, Scott Blum, shalin)
- SOLR-8694: DistributedMap/Queue can create too many Watchers and some code simplification.
(Scott Blum via Mark Miller)
- SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and
SolrCmdDistributor, which violates the lifecycle and can cause bugs.
(hossman via Mark Miller)
- SOLR-8656: PeerSync should use same nUpdates everywhere.
(Ramsey Haddad via Mark Miller)
- SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections from interfering with each other
and other small LeaderElector improvements.
(Scott Blum via Mark Miller, Alan
Woodward)
- SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection
can be made.
(Keith Laban, Dennis Gove)
- SOLR-8420: Fix long overflow in sumOfSquares for Date statistics.
(Tom Hill, Christine Poerschke,
Tomás Fernández Löbbe)
- SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10 even though the thread pool
size is 100. The limit has now been increased to 100.
(Scott Blum, shalin)
- SOLR-8375: ReplicaAssigner rejects valid nodes
(Kelvin Tan, noble)
- SOLR-8738: Fixed false success response when invalid deleteByQuery requests intially hit non-leader
cloud nodes
(hossman)
- SOLR-8771: Multi-threaded core shutdown creates executor per core.
(Mike Drob via Mark Miller)
- SOLR-8145: Fix position of OOM killer script when starting Solr in the background
(Jurian Broertjes via
Timothy Potter)
- SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas that have non-"id"
unique field
(Erik Hatcher, Anshum Gupta)
- SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica
placement. splitshard should preassign nodes using rules, if rules are present
(noble, Shai Erera)