Commit Graph

339 Commits

Author SHA1 Message Date
Eduard Domínguez
63b306d34c GH-1176: KeyValueSerdeResolver improvements
Use extended properties when initializing Consumer and Producer Serdes.

Updated copyright years and authors.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1176
2021-12-10 14:02:29 -05:00
buildmaster
5cd8e06ec6 Bumping versions to 3.2.2-SNAPSHOT after release 2021-12-01 16:58:07 +00:00
buildmaster
79be11c9e9 Going back to snapshots 2021-12-01 16:58:07 +00:00
buildmaster
fc4358ba10 Update SNAPSHOT to 3.2.1 2021-12-01 16:55:37 +00:00
buildmaster
f3d2287b70 Bumping versions to 3.2.1-SNAPSHOT after release 2021-12-01 13:16:23 +00:00
buildmaster
220ae98bcc Going back to snapshots 2021-12-01 13:16:23 +00:00
buildmaster
bd3eebd897 Update SNAPSHOT to 3.2.0 2021-12-01 13:14:14 +00:00
Soby Chacko
ed8683dcc2 KafkaStreams binder health check improvements
Allow health checks on KafkaStreams processors that are currently stopped through
actuator bindings endpoint. Add this only as an opt-in feature through a new binder
level property - includeStoppedProcessorsForHealthCheck which is false by default
to preserve the current health indicator behavior.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1165
Resolves #1175
2021-12-01 10:51:39 +01:00
Pommerening, Nico
aff0dc00ef GH-1161: InteractiveQueryService improvements
This PR safe guards state store instances in case there are multiple KafkaStreams
instances present that have distinct application IDs but share State Store Names.

Change is backwards compatible: In case no KafkaStreams association of the thread
can be found, all local state stores are queried as before.

In case an associated KafkaStreams Instance is found, but required StateStore is
not found in this instance, a warning is issued but backwards compatibility is
preserved by looking up all state stores.

Store within KafkaStreams instance of thread is preferred over "foreign" store with same name.

Warning is issued if requested store is not found within KafkaStreams instance of thread.

The main benefit here is to get rid of randomly selecting stores across all KafkaStreams instances
in case a store is contained within multiple streams instances with same name.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1161
2021-11-18 14:08:22 -05:00
buildmaster
d37cbc79d7 Going back to snapshots 2021-11-03 09:32:29 +00:00
buildmaster
7a03eeed02 Update SNAPSHOT to 3.2.0-RC1 2021-11-03 09:31:02 +00:00
Oleg Zhurakousky
3a88839a5f Disable 'testGlobalStartOffsetWithLatestAndIndividualBindingWthEarliest' temporary 2021-11-03 10:17:03 +01:00
Soby Chacko
c9a07729dd Version updates and compile fixes
Spring Kafka: 2.8.0-RC1
Spring Integration: 5.5.5
Kafka Clients: 3.0.0

Remove schema registry references.
Updates for removed classes/deprecations in Kafka Streams client.
2021-11-01 22:11:06 -04:00
Soby Chacko
07f10f6eb5 Cleaning up
* Disconnect spring-cloud-scheam-registry-client from Kafka Streams binder
  that is used for testing
* Deprecate MessageConverterDelegateSerde
* Remove MessageConverterDelegateSerdeTest
2021-10-28 19:46:46 -04:00
buildmaster
f3a954fad7 Going back to snapshots 2021-10-19 09:29:32 +00:00
buildmaster
7be0f1be23 Update SNAPSHOT to 3.2.0-M3 2021-10-19 09:28:11 +00:00
Soby Chacko
82a3306cb9 GH-1157: Issues with Kafka Streams and Kotlin
Kafka Streams binder erroneously tries to parse regular
non Kafka streams Kotlin function registrations. Ignore
function beans ending in _registration in Kafka Streams binder.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1157
2021-10-01 11:09:58 -04:00
buildmaster
af0485e241 Going back to snapshots 2021-10-01 14:13:49 +00:00
buildmaster
ea8912b011 Update SNAPSHOT to 3.2.0-M2 2021-10-01 14:12:23 +00:00
Soby Chacko
bd1b49222c GH-1156: Kafka Streams binder composition issues
When both regular Kafka and Kafka Streams functions are present,
the code that was added recently for function composition in
Kafka Streams binder was accidentally creating a binadable proxy
factory bean for non Kafka Streams functions. Resolving this issue.

Resovles https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1156
2021-09-28 15:43:06 -04:00
Soby Chacko
9fd16416d6 GH-1149: Kafka Streams global config issues
When there are multiple functions, streamConfigGlobalProperties are overriden
for subsequent functions, after a binding specific config takes effect in a function.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1149
2021-09-24 10:01:26 -04:00
bono007
a4c38b3453 GH-1152: Property binding in Kafka Streams binder
Add default mappings provider for Kafka Streams (move kafka streams default mapping to new provider)

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1152
2021-09-23 13:19:35 -04:00
Soby Chacko
6eed115cc9 Kafka Streams binder tests cleanup 2021-08-27 20:03:56 -04:00
Soby Chacko
8e6d07cc7b Update Kafka Streams branching docs/tests
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1133
2021-08-27 18:16:42 -04:00
Soby Chacko
adde49aab3 Refactoring Kafka Streams join tests
Coalesce the stream join tests in Kafka Streams binder around the functional model.
Remove duplicating the same join tests using the StreamListener model.
2021-08-25 15:45:38 -04:00
Soby Chacko
e500138486 Consumer/Producer prefix in Kafka Streams binder (#1131)
* Consumer/Producer prefix in Kafka Streams binder

Kafka Streams allows the applications to separate the consumer and producer
properties using consumer/producer prefixes. Add these prefixes automatically
if they are missing from the application.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1065

* Addressing PR review comments
2021-08-25 13:57:42 -04:00
Soby Chacko
afe39bf78a Kafka Streams binding lifecycle changes
Start Kafka Streams bindgings only when they are not running.
Similarly, stop them only if they are running.

Without these guards in the bindings for KStream, KTable and GlobalKTable,
it may cause NPE's due to the backing concurrent collections in KafkaStreamsRegistry
not finding the proper KafkaStreams object, especially when the StreamsBuilderFactory
bean is already stopped through the binder provided manager.
2021-08-24 16:53:23 -04:00
Soby Chacko
739b499966 Honoring auto startup in Kafka Streams binder (#1127)
* Honoring auto startup in Kafka Streams binder

When using Kafka Streams bineder, the processors are started
unconditionally, i.e. autoStartup is always true by default.
If spring.kafka.streams.auto-startup is set, then honor that
as the auto-startup flag in the binder.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1126

* Addressing PR review comments

Auto startup flag is honored per individual consumer bindings as well.

* Addressing PR review comments

* Making KafkaStreamsRegistry#kafkaStreams set to use a concurrent Set.
2021-08-23 14:18:05 -04:00
Soby Chacko
f4d3715317 Recycle KafkaStreams Objects
In the event Kafka Streams bindings are restarted (stop/start)
using the actuator bindings endpoints, the underlying KafkaStreams
objects are not recycled. After restarting, it still sees the previous
KafkaStreams object. Addressing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1119
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1120
2021-08-12 15:47:40 -04:00
buildmaster
9b5e735f74 Going back to snapshots 2021-07-30 15:12:01 +00:00
buildmaster
201668542b Update SNAPSHOT to 3.2.0-M1 2021-07-30 15:10:38 +00:00
Soby Chacko
a5f01f9d6f GH-1110: Kafka Streams state machine changes
Restore Kafka Streams error state behavior in the binder, equivalent
to Kafka Streams prior to 2.8. Starting with 2.8, users can customize
the way uncaught errors are interpreted via an UncaughtExceptionHandler
in the applicaiton. Binder now sets a default handler that shuts down
the Kafka Streams client if thre is an uncaught error.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1110
2021-07-29 13:35:08 -04:00
Soby Chacko
912c47e3ac Fix failing tests in Kafka Streams binder
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1109
2021-07-28 18:41:22 -04:00
Soby Chacko
001882de4e Upgrade versions
Spring Kafka: 2.8.0-M1
Spring Integration Kafka: 5.5.2
Kafka: 2.8.0

Ignore a few Kafka Streams binder tests temporarily.
2021-07-27 20:01:08 -04:00
Soby Chacko
54ac274ea3 GH-1085: Allow KTable binding on the outbound
At the moment, Kafka Streams binder only allows KStream bindings on the outbound.
There is a delegation mechanism in which we stil can use KStream for output binding
while allowing the applications to provide a KTable type as the function return type.

Update docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1085

Resolves #1105
2021-07-16 09:46:04 +02:00
Oleg Zhurakousky
b5cb32767b Fix POMs for 3.2 2021-07-13 16:50:01 +02:00
Soby Chacko
7cae3aa54f GH-1096: Named components in Kafka Streams
Support KIP-307 in Kafka Streams binder where the input (source)
and output (sink) bindings are customized with user-provided names.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1096

Resolves #1098
2021-07-05 14:48:09 +02:00
Soby Chacko
5454d54faf Support Kafka Streams binder function composition
Composed function defintions can start with a `java.util.function.Function`
or `java.util.function.BiFunction` and compose with other functions or consumers.
In the case of a consumer, this needs to be the last unit in the function definition.

`java.util.function.BiConumer` is not eligibe for function composition.

The first component in the definition can be a curried function as well.

Adding tests and docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1088

Resolves #1091
2021-07-05 14:29:25 +02:00
Soby Chacko
4d68282937 Refactor JAAS initializer tests 2021-06-23 21:33:11 -04:00
Soby Chacko
9cf4cdc7c1 KafkaStreamsBinderBootstrapTest updates
Migrate tests from StreamListener to the functional model.
2021-06-23 18:30:56 -04:00
Soby Chacko
69a3c36db9 GH-1094: Fix JAAS initializer tests on CI
Before running the JAAS initilizer security tests, remove reference to
JAAS related  config files created previously in other tests. This is done
through clearing a system property (java.security.auth.login.config) in
the tests. Without clearing this property, these tests run into a race condition.

Fixes https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1094
2021-06-23 17:56:38 -04:00
Soby Chacko
b8be381828 Temporarily disabling Kafka Streams jaas test 2021-06-22 21:31:36 -04:00
Soby Chacko
162993dd46 Test fixes for Kafka Streams jass config tests
Provide different application id's.
2021-06-22 21:02:10 -04:00
Soby Chacko
ee3096658a GH-1092: Fix JAAS config issues (Kafka Streams)
Currently, Kafka Streams binder does not honor JAAS configuration properties.
Address this by adding the same `KafkaJaasLoginModuleInitializer` bean used in
regular Kafka binder.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1092
2021-06-22 17:16:33 -04:00
Soby Chacko
6e46054d7b Address deprecations for StreamsBuilderFactoryBeanCustomizer
StreamsBuilderFactoryBeanCustomizer is now renamed to StreamsBuilderFactoryBeanConfigurer
2021-06-08 16:56:47 -04:00
buildmaster
5adeea2acb Bumping versions to 3.1.4-SNAPSHOT after release 2021-05-26 15:50:17 +00:00
buildmaster
65664a2d4c Going back to snapshots 2021-05-26 15:50:17 +00:00
buildmaster
f5e12a29c1 Update SNAPSHOT to 3.1.3 2021-05-26 15:48:31 +00:00
Soby Chacko
9e4a1075d4 Native compilation changes for Kafka Streams binder
This comnmit effectively removes changes introduced by a4ad9e2c0b.
Removing spring.factories mechanism of registering binders in preference to using spring.binders.
2021-04-29 19:35:15 -04:00
Oleg Zhurakousky
7e48afa005 Merge pull request #1067 from sobychacko/kafka-streams-native-1
Native changes required by Kafka Streams binder
2021-04-28 19:13:52 +02:00