362 Commits

Author SHA1 Message Date
buildmaster
2eedc8a218 Update SNAPSHOT to 3.1.0 2020-12-21 12:27:25 +00:00
buildmaster
49d263eafc Going back to snapshots 2020-12-11 15:12:31 +00:00
buildmaster
94f398a3e4 Update SNAPSHOT to 3.1.0-RC1 2020-12-11 15:11:13 +00:00
Soby Chacko
386a361a66 Event type based routing in Kafka Streams binder
Introducing the capability of routing records based on
event types. If a header in the incoming record contains
the event type set on the binding, then the function
associated with that binding gets invoked.

Adding test/docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1003
2020-12-10 21:18:00 -05:00
Soby Chacko
d1c62bbc26 Kafka Streams binder producer/consumerProperties
Fix the issue of Kafka Streams binder level producerProperties
and consumerProperties (...binder.producerProperties and
...binder.consumerProperties), are not taken into consideration.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/997
2020-12-04 17:54:54 -05:00
Soby Chacko
5a7cc9f257 Fix Serde inference issues in Kafka Streams binder
When there are multiple functions present with different outbound target types,
there is an issue of one function overriding the target type of a previous function
in the catalogue where the binder stores the target type information.
This causes problems for the binder initiated Serde inference. Addressing the issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/994
2020-12-03 19:43:07 -05:00
buildmaster
1afb22d65f Going back to snapshots 2020-11-17 16:04:44 +00:00
buildmaster
097fb89d9e Update SNAPSHOT to 3.1.0-M4 2020-11-17 16:03:30 +00:00
Soby Chacko
23ce9e3d6e Fix NPE in Kafka Streams binder
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/981
2020-11-05 12:34:59 -05:00
Soby Chacko
73921db3ec Fixing test failure
Fixing Kafka Streams binder retry tests due to a
wrong conditional check.
2020-11-05 11:05:22 -05:00
Soby Chacko
f1e3a0bdd6 Allow retries in Kafka Streams binder (#980)
* Allow retries in Kafka Streams binder

Provide applications the capability to retry critical sections of the
business logic. This is accomplished through a new API using which
critical path can be wrapped inside a Callable.

Adding tests and docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/945

* Reworking the PR

Remove the binder API that was added before for retrying.
Reuse binding provided RetryTemplate.

Tests and docs

* Cleanup

* Addressing PR review comments

* Fix typo
2020-10-30 16:39:49 -04:00
Soby Chacko
97e3b61d14 Adding improvements to InteractiveQueryService
New API additions.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/942
2020-10-28 18:58:18 -04:00
Soby Chacko
50f4470fcf Update deprecated API usage in InteractiveQueryService
Use queryMetadataForKey instead of metadataForKey

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/941
2020-10-28 15:24:09 -04:00
Soby Chacko
0a0d3a1057 Remove duplicate KafkaStreams topology endpoint
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/895
2020-10-22 12:31:40 -04:00
Soby Chacko
5cdd8a09f9 Custom DLQ Destination Resolver (#976)
* Custom DLQ Destination Resolver

Allow applications to provide a custom DLQ destination resolver
implementaiton by providing a new interface DlqDestinationResolver
as part of binder's public contract. This interface is a BiFunction
extension using which the applications can provide more fine grained
control over where to route records in error.

Adding test to verify.

Adding docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/966

* Add DlqDestinationResolver to MessageChannel based binder.

Tests and docs
2020-10-21 12:02:02 -04:00
Soby Chacko
514530db22 Upgrade dependencies
Spring Kafka -> 2.6.3-SNAPSHOT
Spring Integration Kafka -> 5.4.0-SNAPSHOT
Kafka version -> 2.6.0
Use Kafka_2.13 for tests

Ungignore the Jaas security tests.
Unignore a few Kafka Streams binder tests.
2020-10-16 16:44:32 -04:00
Gary Russell
50ec8f0919 Change Log etc. Replication Factor
If the user has not explicitly set the `SreamsConfig.REPLICATION_FACTOR_CONFIG`,
set it from the binder property.

This is used for infrastructure topics (change logs and repartition topics).
2020-10-15 16:22:41 -04:00
buildmaster
1358bdfeec Going back to snapshots 2020-10-02 14:10:39 +00:00
buildmaster
f8dac888e4 Update SNAPSHOT to 3.1.0-M3 2020-10-02 14:09:20 +00:00
Soby Chacko
50380dae69 Ignore a few Kafka Streams tests temporarily 2020-10-02 09:12:22 -04:00
Oleg Zhurakousky
23b9d5b4c6 Temporary disable failing tests 2020-10-02 12:59:18 +02:00
buildmaster
ec52fbe2eb Going back to snapshots 2020-07-20 16:28:42 +00:00
buildmaster
5e128aafdc Update SNAPSHOT to 3.1.0-M2 2020-07-20 16:27:22 +00:00
Soby Chacko
4159217023 Kafka Streams metrics changes
* Kafka Streams metrics in the binder for Boot 2.2 users are streamlined to
  reflect the Micrometer native support added with version 1.4.0 which is
  available through Boot 2.3. While Boot 2.3 users will get this native support
  from  Micrometer, Boot 2.2 users will still rely on the custom implementation
  in the binder. This commit aligns that custom implemenation more with
  the native implementation.

* Disable the custom Kafka Streams metrics bean which is mentioned above
  (KafkaStreamsBinderMetrics) when the application is on Boot 2.3, as this
  implementation is only applicable for Boot 2.2.x.

* Update docs

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/880
2020-07-17 20:28:47 -04:00
Soby Chacko
dc7c8d657a Fixing javadoc errors
Resovles https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/915
2020-06-16 18:37:30 -04:00
Soby Chacko
a549810899 Renaming Kafka Streams topology actuator endpoint
Kafka Streams topology actuator endpoint had a conflict with the JMX exporter
and was causing some IDE issues. Renming this endpoint to kafkastreamstopology.
Renaming the underlying methods in this actuator endpoint implentation.

Updating docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/895
2020-06-15 18:56:18 -04:00
Gary Russell
95bc54b991 GH-914: Add Micrometer Producer/Consumer Listeners
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/914

Boot no longer uses the deprecated JMX MBean scraping provided by Micrometer.

Add configuration to add Micrometer Meters when Micrometer and spring-kafka 2.5.x are on
the class path.

Micrometer for Streams

- work around until SK 2.5.3
2020-06-15 11:36:49 -04:00
Soby Chacko
21a8f89f22 Update version to 3.1.0-SNAPSHOT 2020-06-02 13:58:54 -04:00
Soby Chacko
7fc9145a21 Fix a failing test due to application ID clash 2020-06-02 12:56:08 -04:00
Soby Chacko
788dc28d7e Simplify the handling of concurrency
In Kafka Streams binder, we use a complex strategy to handle
applications provided concurrency. Simplify this process while
making the binder stay aligned with boot -> binder -> binding
hierrachy for concurrency (num.stream.threads).

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/905
Resolves #907
2020-06-02 12:42:14 -04:00
Soby Chacko
7adb10bcd2 Adding state store beans for KTable binding
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/897
2020-06-02 12:38:14 -04:00
Soby Chacko
8be9fe6abc Ignore test 2020-06-02 12:37:51 -04:00
Soby Chacko
04e0bf628c Kafka Streams binder test triage.
Fixing two tests in Kafka Streams binder that fail with the full suite.
See the comments on the code committed for more details.
2020-06-02 12:37:37 -04:00
Soby Chacko
79323588ba Ignore a test temporarily 2020-06-02 12:37:17 -04:00
Soby Chacko
395683fdaf Fixing random test failure on CI 2020-06-02 12:36:55 -04:00
Soby Chacko
2a25f7c1f3 Cleanup test 2020-06-02 12:36:13 -04:00
Soby Chacko
5fa88e11ec Fix checkstyle 2020-06-02 12:35:45 -04:00
Soby Chacko
ee3279072d Fixing a random test failure on CI 2020-06-02 12:35:20 -04:00
Soby Chacko
fa0d6fe60e GH-899: num.stream.threads not taking effect
Binder is overriding the num.stream.threads property specified
through the binder configuration. Fixing this issue.

Adding tests to verify.

Doc changes

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/899
2020-06-02 12:34:50 -04:00
Soby Chacko
a190289bff Kafka Streams binder changes
Update API calls in health indicator.
Update tests.
Ignore two tests temporarily
2020-04-30 21:42:49 -04:00
Gary Russell
1cfb63f4bf Reject improper settings for bootstrap.servers
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/875

- the `bootstrap.servers` property cannot be overridden at the binding level.
2020-04-30 17:08:03 -04:00
buildmaster
36f00e5867 Going back to snapshots 2020-04-07 17:19:16 +00:00
buildmaster
e7487b9ada Update SNAPSHOT to 3.1.0.M1 2020-04-07 17:18:40 +00:00
Soby Chacko
cc5d1b1aa6 GH:851 Kafka Streams topology visualization
Add Boot actuator endpoints for topology visualization.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/851
2020-03-25 13:02:57 +01:00
Serhii Siryi
65f8cc5660 InteractiveQueryService API changes
* Add InteractiveQueryService.getAllHostsInfo to fetch metadata
   of all instances that handles specific store.
 * Test to verify this new API (in the existing InteractiveQueryService tests).
2020-03-05 13:24:32 -05:00
Soby Chacko
9d212024f8 Updates for 3.1.0
spring-cloud-build to 3.0.0 snapshot
spring kafka to 2.4.x snapshot
SIK 3.2.1

Remove a test that has behavior inconsistent with new changes in Spring Kafka 2.4
where all error handlers have isAckAfterHandle() true by default. The test for
auto commit offset on error without dlq was expecting this acknowledgement to not
to occur. If applications need to have the ack turned off on error, they should provide a
container customizer where it sets the ack to false. Since this is not a binder
concern, we are removing the test testDefaultAutoCommitOnErrorWithoutDlq.

Cleaning up in Kafka Streams binder tests.
2020-03-04 18:37:08 -05:00
Oleg Zhurakousky
e46bd1f844 Update POMs for Ivyland 2020-02-14 11:28:28 +01:00
buildmaster
5a8aad502f Bumping versions to 3.0.3.BUILD-SNAPSHOT after release 2020-02-13 08:31:42 +00:00
buildmaster
b84a0fdfc6 Going back to snapshots 2020-02-13 08:31:41 +00:00
buildmaster
e3fcddeab6 Update SNAPSHOT to 3.0.2.RELEASE 2020-02-13 08:30:55 +00:00