Spring Cloud Stream Source Sample ================================== ## What is this app? This is a Spring Boot application that is a Spring Cloud Stream sample source app that polls a JDBC Source. This application specifically uses MySQL (MariaDB flavor) to query a particular table and output the contents of the table every n seconds(details below). In reality, you probably want to use the out of the box variant of JDBC source from https://github.com/spring-cloud-stream-app-starters/jdbc which is more advanced. Refer to the documentation of Spring Cloud Stream app starters for the latest links for generated apps for specific binders. ## What is the default binder used in this app? The default binder used in this app is Kafka. If you need to change it to Rabbitmq, change the appropriate dependency in the maven pom.xml ## Requirements To run this sample, you will need to have installed: * Java 8 or Above This sample Source uses Spring Integration to poll a JDBC source and requires a Poller. The app provides a basic trigger based Poller which polls every 5 seconds by default. You can change the polling frequency by using the property `jdbc.triggerDelay`. # Additional properties used in this app Refer to the `application.yml` for the defaults used for the following properties. `jdbc.query` - SQL used to query the database. `jdbc.update` - SQL used to skip records already seen (details below) ## Running the application using Kafka binder The following instructions assume that you are running Kafka and MySql as Docker images. * Go to the application root * `docker-compose up -d` * Open another terminal * Ensure that you have the `mysql` CLI tool installed and then use this command: `mysql -u root -p -h 127.0.0.1 -P 3306 sample_mysql_db` `sample_mysql_db` is the name of the database created by the mysql that is running in the docker container. * Go back to the other terminal (root of this sample app `source`) * `./mvnw clean package` When you start the app, it will insert some test data into the database. See the method annotated with`PostConstruct` in the application. * `java -jar target/sample-jdbc-source-0.0.1-SNAPSHOT.jar` The application has a convenient test sink that consumes from the same destination where the source sends data. This test sink will output the data on the console. Alternatively, you can connect to the topic using a Kafka console consumer and watch the output. ## Changing the default behavior of the app. If you want to query the app every second now, and ignore any records that you already saw, there are provisions for that in this sample. * Stop the app * `java -jar target/sample-jdbc-source-0.0.1-SNAPSHOT.jar --jdbc.triggerDelay=1 --jdbc.query="select id, name, tag from test where tag is NULL order by id" --jdbc.update="update test set tag='1' where id in (:id)"` * Now go to your MySQL CLI: Insert some data `insert into test values (20, 'Bob', NULL);` `insert into test values (22, 'Bob', NULL);` Watch the data appear on the application console. Once you are done testing, stop Kafka running in the docker container: `docker-compose down` ## Running the application using Rabbit binder All the instructions above apply here also, but instead of running the default `docker-compose.yml`, use the command below to start a Rabbitmq cluser. * `docker-compose -f docker-compose-rabbit.yml up -d` * `./mvnw clean package -P rabbit-binder` * `java -jar target/sample-jdbc-source-0.0.1-SNAPSHOT.jar` Once you are done testing: `docker-compose -f docker-compose-rabbit.yml down`