362 Commits

Author SHA1 Message Date
Soby Chacko
8eecf03827 Revert "Revert "Update Kafka client version to 3.1.0""
This reverts commit 01dbb49313.
2022-01-27 10:53:27 -05:00
buildmaster
45245f4b92 Going back to snapshots 2022-01-27 15:39:18 +00:00
buildmaster
9922922036 Update SNAPSHOT to 4.0.0-M1 2022-01-27 15:37:22 +00:00
Oleg Zhurakousky
01dbb49313 Revert "Update Kafka client version to 3.1.0"
This reverts commit a25e2ea0b3.
2022-01-27 16:16:10 +01:00
Soby Chacko
a25e2ea0b3 Update Kafka client version to 3.1.0
Test changes
2022-01-25 17:18:04 -05:00
Soby Chacko
ee888a15ba Fixing test 2022-01-18 12:59:47 -05:00
Soby Chacko
417665773c Merge branch '4.x' into main 2022-01-18 11:50:45 -05:00
Soby Chacko
31b91f47e4 Fixing InteractiveQueryService test.
Fixing checkstyle issues.
2022-01-12 13:03:12 -05:00
Soby Chacko
df9d04fd12 Retries for HostInfo in InteractiveQueryService
InteractiveQueryService methods for finding the host info for Kafka Streams
currently throw exceptions if the underlying KafkaStreams are not ready yet.
Introduce a retry mechanism so that the users can control the behaviour of
these methods by providing the following properties.

spring.cloud.stream.kafka.streams.binder.stateStoreRetry.maxAttempts (default 1)
spring.cloud.stream.kafka.streams.binder.stateStoreRetry.backoffPeriod (default 1000 ms).

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1185
2022-01-11 18:57:00 -05:00
Soby Chacko
3770db7844 Retries for HostInfo in InteractiveQueryService
InteractiveQueryService methods for finding the host info for Kafka Streams
currently throw exceptions if the underlying KafkaStreams are not ready yet.
Introduce a retry mechanism so that the users can control the behaviour of
these methods by providing the following properties.

spring.cloud.stream.kafka.streams.binder.stateStoreRetry.maxAttempts (default 1)
spring.cloud.stream.kafka.streams.binder.stateStoreRetry.backoffPeriod (default 1000 ms).

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1185
2022-01-11 18:49:23 -05:00
Eduard Domínguez
e512b7a2c6 Fix: KeySerde setup not using expected key type headers
checkstyle fixes
2022-01-11 14:54:33 -05:00
Eduard Domínguez
406e20f19c Fix: KeySerde setup not using expected key type headers
checkstyle fixes
2022-01-11 14:37:20 -05:00
Soby Chacko
da9bc354e4 StreamListener docs cleanup.
Fixing Kafka Streams composition tests due to an application.id issue.
2022-01-07 20:04:55 -05:00
Soby Chacko
3cc3680f63 Event type routing improvements (Kafka Streams)
When routing by event types, the deserializer omits the
topic and header information. Fixing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1186
2022-01-05 19:33:35 -05:00
Soby Chacko
648188fc6b Event type routing improvements (Kafka Streams)
When routing by event types, the deserializer omits the
topic and header information. Fixing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1186
2022-01-05 19:30:36 -05:00
Soby Chacko
1cdfb962c9 checkstyle fix 2022-01-03 19:00:19 -05:00
Soby Chacko
fb03d2ae8e Version upgrades
4.0.0-SNAPSHOT
  Spring Kafka - 3.0.0-SNAPSHOT
  Spring Integraton Kafka - 6.0.0-SNAPSHOT
  Spring Cloud Stream - 4.0.0-SNAPSHOT

Code changes for Jakarta
2022-01-03 19:00:12 -05:00
Eduard Domínguez
921b47d1e4 GH-1176: KeyValueSerdeResolver improvements
Use extended properties when initializing Consumer and Producer Serdes.

Updated copyright years and authors.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1176
2022-01-03 19:00:05 -05:00
Soby Chacko
be474f643a KafkaStreams binder health check improvements
Allow health checks on KafkaStreams processors that are currently stopped through
actuator bindings endpoint. Add this only as an opt-in feature through a new binder
level property - includeStoppedProcessorsForHealthCheck which is false by default
to preserve the current health indicator behavior.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1165
Resolves #1175
2022-01-03 18:59:57 -05:00
Pommerening, Nico
f9dfbe09f7 GH-1161: InteractiveQueryService improvements
This PR safe guards state store instances in case there are multiple KafkaStreams
instances present that have distinct application IDs but share State Store Names.

Change is backwards compatible: In case no KafkaStreams association of the thread
can be found, all local state stores are queried as before.

In case an associated KafkaStreams Instance is found, but required StateStore is
not found in this instance, a warning is issued but backwards compatibility is
preserved by looking up all state stores.

Store within KafkaStreams instance of thread is preferred over "foreign" store with same name.

Warning is issued if requested store is not found within KafkaStreams instance of thread.

The main benefit here is to get rid of randomly selecting stores across all KafkaStreams instances
in case a store is contained within multiple streams instances with same name.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1161
2022-01-03 18:59:15 -05:00
Eduard Domínguez
63b306d34c GH-1176: KeyValueSerdeResolver improvements
Use extended properties when initializing Consumer and Producer Serdes.

Updated copyright years and authors.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1176
2021-12-10 14:02:29 -05:00
buildmaster
5cd8e06ec6 Bumping versions to 3.2.2-SNAPSHOT after release 2021-12-01 16:58:07 +00:00
buildmaster
79be11c9e9 Going back to snapshots 2021-12-01 16:58:07 +00:00
buildmaster
fc4358ba10 Update SNAPSHOT to 3.2.1 2021-12-01 16:55:37 +00:00
buildmaster
f3d2287b70 Bumping versions to 3.2.1-SNAPSHOT after release 2021-12-01 13:16:23 +00:00
buildmaster
220ae98bcc Going back to snapshots 2021-12-01 13:16:23 +00:00
buildmaster
bd3eebd897 Update SNAPSHOT to 3.2.0 2021-12-01 13:14:14 +00:00
Soby Chacko
ed8683dcc2 KafkaStreams binder health check improvements
Allow health checks on KafkaStreams processors that are currently stopped through
actuator bindings endpoint. Add this only as an opt-in feature through a new binder
level property - includeStoppedProcessorsForHealthCheck which is false by default
to preserve the current health indicator behavior.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1165
Resolves #1175
2021-12-01 10:51:39 +01:00
Pommerening, Nico
aff0dc00ef GH-1161: InteractiveQueryService improvements
This PR safe guards state store instances in case there are multiple KafkaStreams
instances present that have distinct application IDs but share State Store Names.

Change is backwards compatible: In case no KafkaStreams association of the thread
can be found, all local state stores are queried as before.

In case an associated KafkaStreams Instance is found, but required StateStore is
not found in this instance, a warning is issued but backwards compatibility is
preserved by looking up all state stores.

Store within KafkaStreams instance of thread is preferred over "foreign" store with same name.

Warning is issued if requested store is not found within KafkaStreams instance of thread.

The main benefit here is to get rid of randomly selecting stores across all KafkaStreams instances
in case a store is contained within multiple streams instances with same name.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1161
2021-11-18 14:08:22 -05:00
Soby Chacko
486469da51 Kafka binder test migration
- EnableBinding to functional
2021-11-11 16:34:22 -05:00
Soby Chacko
ed98f1129d Kafka Streams binder deprecated component removals 2021-11-09 14:07:23 -05:00
Soby Chacko
c32be995f6 4.0.x changes for Kafka Streams tests
Migrating StreamListener based Kafka Streams binder tests to
use the funcitonal model
2021-11-08 19:21:49 -05:00
buildmaster
d37cbc79d7 Going back to snapshots 2021-11-03 09:32:29 +00:00
buildmaster
7a03eeed02 Update SNAPSHOT to 3.2.0-RC1 2021-11-03 09:31:02 +00:00
Oleg Zhurakousky
3a88839a5f Disable 'testGlobalStartOffsetWithLatestAndIndividualBindingWthEarliest' temporary 2021-11-03 10:17:03 +01:00
Soby Chacko
c9a07729dd Version updates and compile fixes
Spring Kafka: 2.8.0-RC1
Spring Integration: 5.5.5
Kafka Clients: 3.0.0

Remove schema registry references.
Updates for removed classes/deprecations in Kafka Streams client.
2021-11-01 22:11:06 -04:00
Soby Chacko
07f10f6eb5 Cleaning up
* Disconnect spring-cloud-scheam-registry-client from Kafka Streams binder
  that is used for testing
* Deprecate MessageConverterDelegateSerde
* Remove MessageConverterDelegateSerdeTest
2021-10-28 19:46:46 -04:00
buildmaster
f3a954fad7 Going back to snapshots 2021-10-19 09:29:32 +00:00
buildmaster
7be0f1be23 Update SNAPSHOT to 3.2.0-M3 2021-10-19 09:28:11 +00:00
Soby Chacko
82a3306cb9 GH-1157: Issues with Kafka Streams and Kotlin
Kafka Streams binder erroneously tries to parse regular
non Kafka streams Kotlin function registrations. Ignore
function beans ending in _registration in Kafka Streams binder.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1157
2021-10-01 11:09:58 -04:00
buildmaster
af0485e241 Going back to snapshots 2021-10-01 14:13:49 +00:00
buildmaster
ea8912b011 Update SNAPSHOT to 3.2.0-M2 2021-10-01 14:12:23 +00:00
Soby Chacko
bd1b49222c GH-1156: Kafka Streams binder composition issues
When both regular Kafka and Kafka Streams functions are present,
the code that was added recently for function composition in
Kafka Streams binder was accidentally creating a binadable proxy
factory bean for non Kafka Streams functions. Resolving this issue.

Resovles https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1156
2021-09-28 15:43:06 -04:00
Soby Chacko
9fd16416d6 GH-1149: Kafka Streams global config issues
When there are multiple functions, streamConfigGlobalProperties are overriden
for subsequent functions, after a binding specific config takes effect in a function.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1149
2021-09-24 10:01:26 -04:00
bono007
a4c38b3453 GH-1152: Property binding in Kafka Streams binder
Add default mappings provider for Kafka Streams (move kafka streams default mapping to new provider)

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1152
2021-09-23 13:19:35 -04:00
Soby Chacko
6eed115cc9 Kafka Streams binder tests cleanup 2021-08-27 20:03:56 -04:00
Soby Chacko
8e6d07cc7b Update Kafka Streams branching docs/tests
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1133
2021-08-27 18:16:42 -04:00
Soby Chacko
adde49aab3 Refactoring Kafka Streams join tests
Coalesce the stream join tests in Kafka Streams binder around the functional model.
Remove duplicating the same join tests using the StreamListener model.
2021-08-25 15:45:38 -04:00
Soby Chacko
e500138486 Consumer/Producer prefix in Kafka Streams binder (#1131)
* Consumer/Producer prefix in Kafka Streams binder

Kafka Streams allows the applications to separate the consumer and producer
properties using consumer/producer prefixes. Add these prefixes automatically
if they are missing from the application.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1065

* Addressing PR review comments
2021-08-25 13:57:42 -04:00
Soby Chacko
afe39bf78a Kafka Streams binding lifecycle changes
Start Kafka Streams bindgings only when they are not running.
Similarly, stop them only if they are running.

Without these guards in the bindings for KStream, KTable and GlobalKTable,
it may cause NPE's due to the backing concurrent collections in KafkaStreamsRegistry
not finding the proper KafkaStreams object, especially when the StreamsBuilderFactory
bean is already stopped through the binder provided manager.
2021-08-24 16:53:23 -04:00