904 Commits

Author SHA1 Message Date
Soby Chacko
3ff99da014 Update parent s-c-build to 2.2.1.RELEASE 2019-12-18 12:07:39 -05:00
Soby Chacko
c7c05fe3c2 Docs for topic patterns in Kafka Streams binder 2019-12-17 18:43:45 -05:00
문찬용
4b50f7c2f5 Fix typo 2019-12-17 18:41:13 -05:00
stoetti
88c5aa0969 Supporting topic pattern in Kafka Streams binder
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/820

Fix checkstyle issues
2019-12-17 18:38:36 -05:00
Oleg Zhurakousky
d5a72a29d0 GH-815 polishing
Resolves #816
2019-12-17 10:55:47 +01:00
Soby Chacko
8c3cb8230b Addressing mulit binder issues with Kafka Streams
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/815

There was an issue with Kafka Streams multi binders in which the properties were not scanned
properly. The last configuation is always won, wiping out any prevous enviroment properties.
Addressing this issue by properly keeping KafkaBinderConfigurationProperties per multi binder
environment and explicity invoking Boot properties binding on them.

Adding test to verify.
2019-12-12 18:08:53 -05:00
Soby Chacko
70c1ce8970 Update image for spring.io kafka streams blog 2019-12-02 11:22:39 -05:00
Soby Chacko
1d8a4e67d2 Update image for spring.io kafka streams blog 2019-12-02 11:16:29 -05:00
buildmaster
bbfc776395 Bumping versions to 3.0.1.BUILD-SNAPSHOT after release 2019-11-22 14:51:01 +00:00
buildmaster
4e9ed30948 Going back to snapshots 2019-11-22 14:51:01 +00:00
buildmaster
34fd7a6a7a Update SNAPSHOT to 3.0.0.RELEASE v3.0.0.RELEASE 2019-11-22 14:49:08 +00:00
Oleg Zhurakousky
b23b42d874 Ignoring intermittently failing test 2019-11-22 15:36:23 +01:00
Soby Chacko
cf59cfcf12 Kafka Streams docs cleanup 2019-11-20 18:25:12 -05:00
Soby Chacko
02a4fcb144 AdminClient caching in KafkaStreams health check 2019-11-20 17:36:14 -05:00
Oleksii Mukas
6ac9c0ed23 Closing adminClient prematurely
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/803
2019-11-20 17:34:09 -05:00
Soby Chacko
78ff4f1a70 KafkaBindingProperties has no documentation (#802)
* KafkaBindingProperties has no documentation

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/739

No popup help in IDE due to missing javadocs in KafkaBindingProperties,
KafkaConsumerProperties and KafkaProducerProperties.

Some IDEs currently only show javadocs on getter methods from POJOs
used insdide a map. Therefore, some javadocs are duplicated between
fields and getter methods.

* Addressing PR review comments

* Addressing PR review comments
2019-11-14 12:50:15 -05:00
Soby Chacko
637ec2e55d Kafka Streams binder docs improvements
Adding docs for production exception handlers.
Updating configuration options section.
2019-11-13 15:19:41 -05:00
Soby Chacko
6effd58406 Adding docs for global state stores 2019-11-13 11:19:36 -05:00
Soby Chacko
7b8f0dcab7 Kafka Streams - DLQ control per consumer binding (#801)
* Kafka Streams - DLQ control per consumer binding

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/800

* Fine-grained DLQ control and deserialization exception handlers per input binding

* Deprecate KafkaStreamsBinderConfigurationProperties.SerdeError in preference to the
  new enum `KafkaStreamsBinderConfigurationProperties.DeserializationExceptionHandler`
  based properties

* Add tests, modifying docs

* Addressing PR review comments
2019-11-13 09:41:02 -05:00
Gary Russell
88912b8d6b GH-715: Add retry, dlq for transactional binders
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/715

When using transactions, binder retry, dlq cannot be used because the retry
runs within the transaction, which is undesirable if there is another resource
involved; also publishing to the DLQ could be rolled back.

Use the retry properties to configure an `AfterRollbackProcessor` to perform the
retry and DLQ publishing after the transaction has rolled back.

Add docs; move the container customizer invocation to the end.
2019-11-12 14:53:12 -05:00
Soby Chacko
34b0945d43 Changing the order of calling customizer
In the Kafka Streams binder, StreamsBuilderFactoryBean customzier was being called
prematurely before the object is created. Fixing this issue.

Add a test to verify
2019-11-11 17:21:46 -05:00
Gary Russell
278ba795d0 Configure consumer customizer
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/1841

* Add test
2019-11-11 11:34:41 -05:00
buildmaster
bf30ecdea1 Going back to snapshots 2019-11-08 16:29:53 +00:00
buildmaster
464ce685bb Update SNAPSHOT to 3.0.0.RC2 v3.0.0.RC2 2019-11-08 16:29:19 +00:00
Oleg Zhurakousky
54fa9a638d Added flatten plug-in 2019-11-08 16:50:10 +01:00
Soby Chacko
fefd9a3bd6 Cleanup in Kafka Streams docs 2019-11-07 18:07:27 -05:00
Soby Chacko
8e26d5e170 Fix typos in the previous PR commit 2019-11-06 21:23:19 -05:00
Soby Chacko
ac6bdc976e Reintroduce BinderHeaderMapper (#797)
* Reintrouce BinderHeaderMapper

Provide a  custom header mapper that is identical to the DefaultKafkaHeaderMapper in Spring Kafka.
This is to address some interoperability issues between Spring Cloud Stream 3.0.x and 2.x apps,
where mime types in the header are not de-serialized properly. This custom BinderHeaderMapper
will be eventually deprecated and removed once the fixes are in Spring Kafka.

Resolves #796

* Addressing review

* polishing
2019-11-06 21:14:20 -05:00
Ramesh Sharma
65386f6967 Fixed log message to print as value vs key serde 2019-11-06 09:43:08 -05:00
Oleg Zhurakousky
81c453086a Merge pull request #794 from sobychacko/gh-752
Revise docs
2019-11-06 09:11:16 +01:00
Soby Chacko
0ddd9f8f64 Revise docs
Update kafka-clients version
Revise Kafka Streams docs

Resolves #752
2019-11-05 19:49:07 -05:00
Gary Russell
d0fe596a9e GH-790: Upgrade SIK to 3.2.1
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/790
2019-11-05 11:41:40 -05:00
Soby Chacko
062bbc1cc3 Add null check for KafkaStreamsBinderMetrics 2019-11-01 18:42:30 -04:00
Soby Chacko
bc1936eb28 KafkaStreams binder metrics - duplicate entries
When the same metric name is repeated, there are some registry implementations such as the
micrometer Prometheus registry fail to register the duplicate entry. Fixing this issue by
restricting the duplicate metric names not to be registered.

Also, address an issue with multiple processors and metrics in the same application
by prepending the application ID of the Kafka Streams processor in the metric name itself.

Resolves #788
2019-11-01 15:55:30 -04:00
Soby Chacko
e2f1092173 Kafka Streams binder health indicator improvements
Caching AdminClient in Kafk Streams binder health indicator.

Resolves #791
2019-10-31 16:15:56 -04:00
Soby Chacko
06e5739fbd Addressing bugs reported by Sonarqube
Resolves #747
2019-10-25 16:42:02 -04:00
Soby Chacko
ad8e67fdc5 Ignore a test where there is a race condition 2019-10-24 12:00:22 -04:00
Soby Chacko
a3fb4cc3b3 Fix state store registration issue
Fixing the issue where state store is registered for each input binding
when multiple input bindings are present.

Resolves #785
2019-10-24 11:33:31 -04:00
Soby Chacko
7f09baf72d Enable customization on StreamsBuilderFactoryBean
Spring Kafka provides a StreamsBuilderFactoryBeanCustomizer. Use this in the binder so that the
applicatons can plugin in such a bean to further customize the StreamsBuilderFactoryBean and KafkaStreams.

Resolves #784
2019-10-24 09:35:37 -04:00
Soby Chacko
28a02cda4f Multiple functions and definition property
In order to make Kafka Streams binder based function apps more consistent
with the wider functional support in ScSt, it should require the proprety
spring.cloud.stream.fucntion.definition to signal which functions to activate.

Resolves #783
2019-10-24 01:01:34 -04:00
Soby Chacko
f96a9f884c Custom partitioner for Kafka Streams producer
* Allow the ability to plug in custom StreamPartitioner on the Kafka Streams producer.
* Fix a bug where the overriding of native encoding/decoding settings by the binder was not
  workging properly. This fix is done by providing a custom ConfigurationPropertiesBindHandlerAdvisor.
* Add test to verify

Resolves #782
2019-10-23 20:12:08 -04:00
Soby Chacko
a9020368e5 Remove the usage of BindingProvider 2019-10-23 16:32:07 -04:00
Gary Russell
ca9296dbd2 GH-628: Add dlqPartitions property
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/628

Allow users to specify the number of partitions in the dead letter topic.

If the property is set to 1 and the `minPartitionCount` binder property is 1,
override the default behavior and always publish to partition 0.
2019-10-23 15:33:20 -04:00
Soby Chacko
e8d202404b Custom timestamp extractor per binding
Currenlty there is no way to pass a custom timestamp extractor
per consumer binding in Kafka Streams binder. Adding this ability.

Resolves #640
2019-10-23 15:31:17 -04:00
Artem Bilan
5794fb983c Add ProducerMessageHandlerCustomizer support
Related to https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit/issues/265
and https://github.com/spring-cloud/spring-cloud-stream/pull/1828
2019-10-23 14:30:20 -04:00
Oleg Zhurakousky
1ce1d7918f Merge pull request #777 from sobychacko/gh-774
Null value in outbound KStream
2019-10-23 16:54:21 +02:00
Oleg Zhurakousky
1264434871 Merge pull request #773 from sobychacko/gh-552
Kafka/Streams binder health indicator improvements
2019-10-23 16:53:37 +02:00
Massimiliano Poggi
e4fed0a52d Added qualifiers to CompositeMessageConverterBean injections
When more than one CompositeMessageConverter bean was defined in the same ApplicationContext,
not having the qualifiers on the injection points was causing the application to fail during
instantiation due to bean conflicts being raised. The injection points for CompositeMessageConverter
have been marked with the appropriate qualifier to inject the Spring Cloud CompositeMessageConverter.

Resolves #775
2019-10-22 17:17:03 -04:00
Soby Chacko
05e2918bc0 Addressing PR review comments 2019-10-22 17:07:51 -04:00
Gary Russell
6dadf0c104 GH-763: Add DlqPartitionFunction
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/763

Allow users to select the DLQ partition based on

* consumer group
* consumer record (which includes the original topic/partition)
* exception

Kafka Streams DLQ docs changes
2019-10-22 16:10:29 -04:00