Compare commits

..

35 Commits

Author SHA1 Message Date
Christoph Strobl
d94d273010 Fix test 2023-01-25 15:05:56 +01:00
Christoph Strobl
b9f6463337 decrypt? 2023-01-25 14:32:38 +01:00
Christoph Strobl
095022e71d reactive FLE encryptiion works -> next decrypt 2023-01-25 14:32:35 +01:00
Christoph Strobl
329b4b2881 Hacking - Reactive FLE
experiment with resolving reactive types in document
2023-01-25 14:24:35 +01:00
Christoph Strobl
73aeb7a425 Test encryption during update 2023-01-25 14:24:35 +01:00
Christoph Strobl
4b8ac4d249 Some changes that allow reading the alt key from a field
typically only supported in automatic schema but neat to have it here as well. eg. for customer data cyper based on eg. username.
Also make sure to translate decryption exceptions.
2023-01-25 14:24:35 +01:00
Christoph Strobl
1a7157fa7c Encrypt collection of complex types. 2023-01-25 14:24:35 +01:00
Christoph Strobl
10a089fe77 Encrypt collection of simple values 2023-01-25 14:24:35 +01:00
Christoph Strobl
7b93379165 Enable full encryption of nested documents. 2023-01-25 14:24:35 +01:00
Christoph Strobl
0361c3acc9 Hacking 2023-01-25 14:24:35 +01:00
Christoph Strobl
a6641e0c01 Prepare issue branch. 2023-01-25 14:24:34 +01:00
Mark Paluch
33902b5061 Polishing.
Move QuerydslPredicateExecutor hints to RepositoryRuntimeHints.

See #4244
Original pull request: #4245
2023-01-23 14:08:43 +01:00
Christoph Strobl
d00db4bd40 Add missing hints for Querydsl integration.
This commit adds missing reflection configuration for Querydsl integration. We now also make sure to call the queryMixing getter instead of reading the field via reflection.

Closes #4244
Original pull request: #4245
2023-01-23 14:08:43 +01:00
Christoph Strobl
a5dcbf043a Update links in reference documentation.
We now use the springDocsUrl attribute provided via spring-projects/spring-data-build#1895 to resolve links to framework documentation.

Original Pull Request: #4267
2023-01-18 14:23:22 +01:00
robeatoz
c31203582f Fix parameter and method name in reference documentation.
Closes: #4247
2023-01-16 11:22:29 +01:00
Emre Uygun
f146afecdc Fix typo in reference documentation.
Closes: #4250
2023-01-16 11:19:32 +01:00
Christoph Strobl
324a541a64 Polishing.
Original Pull Request: #4255
2023-01-16 11:17:49 +01:00
Michael Krog
6b71d773d7 Fixes return in Javadoc.
Closes: #4255
2023-01-16 11:15:14 +01:00
Patouche
10447afe0c Fix typo in reference documentation.
Closes: #4268
2023-01-16 10:49:42 +01:00
soumyaPrakashB
c9dfd60f0f Add missing Nullable annotation.
For one of constructor arguments of the AggregationOptions the Nullable annotation for the cursor argument is missing.

Closes: #4256
2023-01-16 10:47:28 +01:00
Mark Paluch
26a8fafd03 Upgrade to MongoDB driver 4.8.2.
Closes #4270
2023-01-13 10:30:01 +01:00
Mark Paluch
00f652a094 Polishing.
Add missing package-info.

See #4248
Original pull request: #4249
2023-01-12 08:47:16 +01:00
Christoph Strobl
d050ae5732 Exclude mongodb and data.mongodb namespaces from reflection contribution.
In some cases the users domain model may hold references to spring data or MongoDB specific types which should not be included in the reflection configuration as they are part of the static runtime hints configuration.

Closes #4248
Original pull request: #4249
2023-01-12 08:47:16 +01:00
Christoph Strobl
8bcab93588 Avoid multiple mapping iterations.
A 2nd pass is no longer needed as the context already does all the work.

Closes: #4043
Original pull request: #4240
2023-01-11 16:04:36 +01:00
Mark Paluch
1839f55055 Polishing.
Introduce HintFunction to encapsulate how hints are applied and to remove code duplications.

See #4238
Original pull request: #4243
2023-01-11 16:02:14 +01:00
Christoph Strobl
4220df5bf8 Accept index names as hint for aggregations.
Closes #4238
Original pull request: #4243
2023-01-11 16:02:05 +01:00
Christoph Strobl
95c6d1531f Fix invalid format specifier in debug statement.
Closes #4241
Original pull request: #4246
2023-01-11 15:29:22 +01:00
Christoph Strobl
b7ed099e06 Update broken links in reference documentation.
Original Pull Request: #4267
2023-01-11 13:46:50 +01:00
Maksymilian Babarowski
7e2e546e55 Update links to Spring Framework reference docs.
Closes: #4267
2023-01-11 13:46:38 +01:00
yangwenjie008
7ce2ebe26e Fix class loader issue with LazyLoadingProxyInterceptor.
Restore original behaviour that was unintentionally changed by modifications related to #4148.

Closes: #4260
Original Pull Request: #4261
2023-01-11 08:49:38 +01:00
Mark Paluch
fbf4d1baa8 Extend license header copyright years to 2023.
See #4264
2023-01-02 09:53:33 +01:00
Christoph Strobl
187f260fe4 Upgrade to MongoDB driver 4.8.1
Closes: #4251
2022-12-12 13:45:20 +01:00
Mark Paluch
04411075b4 Update CI properties.
See #4235
2022-11-18 15:31:10 +01:00
Mark Paluch
459a9c191b After release cleanups.
See #4209
2022-11-18 14:30:20 +01:00
Mark Paluch
137cba8bbb Prepare next development iteration.
See #4209
2022-11-18 14:30:19 +01:00
34 changed files with 1487 additions and 128 deletions

2
Jenkinsfile vendored
View File

@@ -9,7 +9,7 @@ pipeline {
triggers {
pollSCM 'H/10 * * * *'
upstream(upstreamProjects: "spring-data-commons/3.0.x", threshold: hudson.model.Result.SUCCESS)
upstream(upstreamProjects: "spring-data-commons/main", threshold: hudson.model.Result.SUCCESS)
}
options {

10
pom.xml
View File

@@ -5,7 +5,7 @@
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.0.1</version>
<version>4.1.x-MANUAL-ENCRYPTION-SNAPSHOT</version>
<packaging>pom</packaging>
<name>Spring Data MongoDB</name>
@@ -15,7 +15,7 @@
<parent>
<groupId>org.springframework.data.build</groupId>
<artifactId>spring-data-parent</artifactId>
<version>3.0.1</version>
<version>3.1.0-SNAPSHOT</version>
</parent>
<modules>
@@ -26,7 +26,7 @@
<properties>
<project.type>multi</project.type>
<dist.id>spring-data-mongodb</dist.id>
<springdata.commons>3.0.1</springdata.commons>
<springdata.commons>3.1.0-SNAPSHOT</springdata.commons>
<mongo>4.8.2</mongo>
<mongo.reactivestreams>${mongo}</mongo.reactivestreams>
<jmh.version>1.19</jmh.version>
@@ -145,8 +145,8 @@
<repositories>
<repository>
<id>spring-libs-release</id>
<url>https://repo.spring.io/libs-release</url>
<id>spring-libs-snapshot</id>
<url>https://repo.spring.io/libs-snapshot</url>
<snapshots>
<enabled>true</enabled>
</snapshots>

View File

@@ -7,7 +7,7 @@
<parent>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.0.1</version>
<version>4.1.x-MANUAL-ENCRYPTION-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

View File

@@ -15,7 +15,7 @@
<parent>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.0.1</version>
<version>4.1.x-MANUAL-ENCRYPTION-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

View File

@@ -13,7 +13,7 @@
<parent>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.0.1</version>
<version>4.1.x-MANUAL-ENCRYPTION-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
@@ -112,6 +112,13 @@
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-crypt</artifactId>
<version>1.6.1</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>

View File

@@ -62,13 +62,14 @@ class MongoRuntimeHints implements RuntimeHintsRegistrar {
TypeReference.of(ReactiveAfterSaveCallback.class)),
builder -> builder.withMembers(MemberCategory.INVOKE_DECLARED_CONSTRUCTORS,
MemberCategory.INVOKE_PUBLIC_METHODS));
}
}
private static void registerTransactionProxyHints(RuntimeHints hints, @Nullable ClassLoader classLoader) {
if (MongoAotPredicates.isSyncClientPresent(classLoader) && ClassUtils.isPresent("org.springframework.aop.SpringProxy", classLoader)) {
if (MongoAotPredicates.isSyncClientPresent(classLoader)
&& ClassUtils.isPresent("org.springframework.aop.SpringProxy", classLoader)) {
hints.proxies().registerJdkProxy(TypeReference.of("com.mongodb.client.MongoDatabase"),
TypeReference.of("org.springframework.aop.SpringProxy"),
@@ -78,4 +79,5 @@ class MongoRuntimeHints implements RuntimeHintsRegistrar {
TypeReference.of("org.springframework.core.DecoratingProxy"));
}
}
}

View File

@@ -0,0 +1,102 @@
/*
* Copyright 2023 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.mongodb.core;
import java.util.function.Function;
import org.bson.conversions.Bson;
import org.springframework.data.mongodb.CodecRegistryProvider;
import org.springframework.data.mongodb.util.BsonUtils;
import org.springframework.lang.Nullable;
import org.springframework.util.StringUtils;
/**
* Function object to apply a query hint. Can be an index name or a BSON document.
*
* @author Mark Paluch
* @since 4.1
*/
class HintFunction {
private static final HintFunction EMPTY = new HintFunction(null);
private final @Nullable Object hint;
private HintFunction(@Nullable Object hint) {
this.hint = hint;
}
/**
* Return an empty hint function.
*
* @return
*/
static HintFunction empty() {
return EMPTY;
}
/**
* Create a {@link HintFunction} from a {@link Bson document} or {@link String index name}.
*
* @param hint
* @return
*/
static HintFunction from(@Nullable Object hint) {
return new HintFunction(hint);
}
/**
* Return whether a hint is present.
*
* @return
*/
public boolean isPresent() {
return (hint instanceof String hintString && StringUtils.hasText(hintString)) || hint instanceof Bson;
}
/**
* Apply the hint to consumers depending on the hint format.
*
* @param registryProvider
* @param stringConsumer
* @param bsonConsumer
* @return
* @param <R>
*/
public <R> R apply(@Nullable CodecRegistryProvider registryProvider, Function<String, R> stringConsumer,
Function<Bson, R> bsonConsumer) {
if (!isPresent()) {
throw new IllegalStateException("No hint present");
}
if (hint instanceof Bson bson) {
return bsonConsumer.apply(bson);
}
if (hint instanceof String hintString) {
if (BsonUtils.isJsonDocument(hintString)) {
return bsonConsumer.apply(BsonUtils.parse(hintString, registryProvider));
}
return stringConsumer.apply(hintString);
}
throw new IllegalStateException(
"Unable to read hint of type %s".formatted(hint != null ? hint.getClass() : "null"));
}
}

View File

@@ -68,6 +68,8 @@ public class MongoExceptionTranslator implements PersistenceExceptionTranslator
private static final Set<String> DATA_INTEGRITY_EXCEPTIONS = new HashSet<>(
Arrays.asList("WriteConcernException", "MongoWriteException", "MongoBulkWriteException"));
private static final Set<String> SECURITY_EXCEPTIONS = Set.of("MongoCryptException");
@Nullable
public DataAccessException translateExceptionIfPossible(RuntimeException ex) {
@@ -131,6 +133,8 @@ public class MongoExceptionTranslator implements PersistenceExceptionTranslator
return new ClientSessionException(ex.getMessage(), ex);
} else if (MongoDbErrorCodes.isTransactionFailureCode(code)) {
return new MongoTransactionException(ex.getMessage(), ex);
} else if(ex.getCause() != null && SECURITY_EXCEPTIONS.contains(ClassUtils.getShortName(ex.getCause().getClass()))) {
return new PermissionDeniedDataAccessException(ex.getMessage(), ex);
}
return new UncategorizedMongoDbException(ex.getMessage(), ex);

View File

@@ -30,7 +30,6 @@ import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.bson.Document;
import org.bson.conversions.Bson;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
@@ -70,16 +69,7 @@ import org.springframework.data.mongodb.core.aggregation.AggregationOptions;
import org.springframework.data.mongodb.core.aggregation.AggregationPipeline;
import org.springframework.data.mongodb.core.aggregation.AggregationResults;
import org.springframework.data.mongodb.core.aggregation.TypedAggregation;
import org.springframework.data.mongodb.core.convert.DbRefResolver;
import org.springframework.data.mongodb.core.convert.DefaultDbRefResolver;
import org.springframework.data.mongodb.core.convert.JsonSchemaMapper;
import org.springframework.data.mongodb.core.convert.MappingMongoConverter;
import org.springframework.data.mongodb.core.convert.MongoConverter;
import org.springframework.data.mongodb.core.convert.MongoCustomConversions;
import org.springframework.data.mongodb.core.convert.MongoJsonSchemaMapper;
import org.springframework.data.mongodb.core.convert.MongoWriter;
import org.springframework.data.mongodb.core.convert.QueryMapper;
import org.springframework.data.mongodb.core.convert.UpdateMapper;
import org.springframework.data.mongodb.core.convert.*;
import org.springframework.data.mongodb.core.index.IndexOperations;
import org.springframework.data.mongodb.core.index.IndexOperationsProvider;
import org.springframework.data.mongodb.core.index.MongoMappingEventPublisher;
@@ -100,7 +90,6 @@ import org.springframework.data.mongodb.core.query.UpdateDefinition;
import org.springframework.data.mongodb.core.query.UpdateDefinition.ArrayFilter;
import org.springframework.data.mongodb.core.timeseries.Granularity;
import org.springframework.data.mongodb.core.validation.Validator;
import org.springframework.data.mongodb.util.BsonUtils;
import org.springframework.data.projection.EntityProjection;
import org.springframework.data.util.CloseableIterator;
import org.springframework.data.util.Optionals;
@@ -117,16 +106,7 @@ import com.mongodb.ClientSessionOptions;
import com.mongodb.MongoException;
import com.mongodb.ReadPreference;
import com.mongodb.WriteConcern;
import com.mongodb.client.AggregateIterable;
import com.mongodb.client.ClientSession;
import com.mongodb.client.DistinctIterable;
import com.mongodb.client.FindIterable;
import com.mongodb.client.MapReduceIterable;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
import com.mongodb.client.MongoIterable;
import com.mongodb.client.*;
import com.mongodb.client.model.*;
import com.mongodb.client.result.DeleteResult;
import com.mongodb.client.result.UpdateResult;
@@ -159,6 +139,7 @@ import com.mongodb.client.result.UpdateResult;
* @author Yadhukrishna S Pai
* @author Anton Barkan
* @author Bartłomiej Mazur
* @author Michael Krog
*/
public class MongoTemplate implements MongoOperations, ApplicationContextAware, IndexOperationsProvider {
@@ -634,7 +615,8 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
}
@Override
public MongoCollection<Document> createView(String name, Class<?> source, AggregationPipeline pipeline, @Nullable ViewOptions options) {
public MongoCollection<Document> createView(String name, Class<?> source, AggregationPipeline pipeline,
@Nullable ViewOptions options) {
return createView(name, getCollectionName(source),
queryOperations.createAggregation(Aggregation.newAggregation(source, pipeline.getOperations()), source),
@@ -642,7 +624,8 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
}
@Override
public MongoCollection<Document> createView(String name, String source, AggregationPipeline pipeline, @Nullable ViewOptions options) {
public MongoCollection<Document> createView(String name, String source, AggregationPipeline pipeline,
@Nullable ViewOptions options) {
return createView(name, source,
queryOperations.createAggregation(Aggregation.newAggregation(pipeline.getOperations()), (Class<?>) null),
@@ -654,7 +637,8 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
return doCreateView(name, source, aggregation.getAggregationPipeline(), options);
}
protected MongoCollection<Document> doCreateView(String name, String source, List<Document> pipeline, @Nullable ViewOptions options) {
protected MongoCollection<Document> doCreateView(String name, String source, List<Document> pipeline,
@Nullable ViewOptions options) {
CreateViewOptions viewOptions = new CreateViewOptions();
if (options != null) {
@@ -2065,7 +2049,10 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
}
options.getComment().ifPresent(aggregateIterable::comment);
options.getHint().ifPresent(aggregateIterable::hint);
HintFunction hintFunction = options.getHintObject().map(HintFunction::from).orElseGet(HintFunction::empty);
if (hintFunction.isPresent()) {
aggregateIterable = hintFunction.apply(mongoDbFactory, aggregateIterable::hintString, aggregateIterable::hint);
}
if (options.hasExecutionTimeLimit()) {
aggregateIterable = aggregateIterable.maxTime(options.getMaxTime().toMillis(), TimeUnit.MILLISECONDS);
@@ -2124,7 +2111,10 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
}
options.getComment().ifPresent(cursor::comment);
options.getHint().ifPresent(cursor::hint);
HintFunction hintFunction = options.getHintObject().map(HintFunction::from).orElseGet(HintFunction::empty);
if (options.getHintObject().isPresent()) {
cursor = hintFunction.apply(mongoDbFactory, cursor::hintString, cursor::hint);
}
Class<?> domainType = aggregation instanceof TypedAggregation ? ((TypedAggregation) aggregation).getInputType()
: null;
@@ -2357,8 +2347,9 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
* @param query the query document that specifies the criteria used to find a record.
* @param fields the document that specifies the fields to be returned.
* @param entityClass the parameterized type of the returned list.
* @return the {@link List} of converted objects.
* @return the converted object or {@literal null} if none exists.
*/
@Nullable
protected <T> T doFindOne(String collectionName, Document query, Document fields, Class<T> entityClass) {
return doFindOne(collectionName, query, fields, CursorPreparer.NO_OP_PREPARER, entityClass);
}
@@ -2372,9 +2363,10 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
* @param fields the document that specifies the fields to be returned.
* @param entityClass the parameterized type of the returned list.
* @param preparer the preparer used to modify the cursor on execution.
* @return the {@link List} of converted objects.
* @return the converted object or {@literal null} if none exists.
* @since 2.2
*/
@Nullable
@SuppressWarnings("ConstantConditions")
protected <T> T doFindOne(String collectionName, Document query, Document fields, CursorPreparer preparer,
Class<T> entityClass) {
@@ -3152,8 +3144,9 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
.ifPresent(cursorToUse::collation);
Meta meta = query.getMeta();
HintFunction hintFunction = HintFunction.from(query.getHint());
if (query.getSkip() <= 0 && query.getLimit() <= 0 && ObjectUtils.isEmpty(query.getSortObject())
&& !StringUtils.hasText(query.getHint()) && !meta.hasValues() && !query.getCollation().isPresent()) {
&& !hintFunction.isPresent() && !meta.hasValues() && !query.getCollation().isPresent()) {
return cursorToUse;
}
@@ -3169,15 +3162,8 @@ public class MongoTemplate implements MongoOperations, ApplicationContextAware,
cursorToUse = cursorToUse.sort(sort);
}
if (StringUtils.hasText(query.getHint())) {
String hint = query.getHint();
if (BsonUtils.isJsonDocument(hint)) {
cursorToUse = cursorToUse.hint(BsonUtils.parse(hint, mongoDbFactory));
} else {
cursorToUse = cursorToUse.hintString(hint);
}
if (hintFunction.isPresent()) {
cursorToUse = hintFunction.apply(mongoDbFactory, cursorToUse::hintString, cursorToUse::hint);
}
if (meta.hasValues()) {

View File

@@ -60,7 +60,6 @@ import org.springframework.data.projection.EntityProjection;
import org.springframework.data.util.Lazy;
import org.springframework.lang.Nullable;
import org.springframework.util.ClassUtils;
import org.springframework.util.StringUtils;
import com.mongodb.client.model.CountOptions;
import com.mongodb.client.model.DeleteOptions;
@@ -567,14 +566,11 @@ class QueryOperations {
if (query.getSkip() > 0) {
options.skip((int) query.getSkip());
}
if (StringUtils.hasText(query.getHint())) {
String hint = query.getHint();
if (BsonUtils.isJsonDocument(hint)) {
options.hint(BsonUtils.parse(hint, codecRegistryProvider));
} else {
options.hintString(hint);
}
HintFunction hintFunction = HintFunction.from(query.getHint());
if (hintFunction.isPresent()) {
options = hintFunction.apply(codecRegistryProvider, options::hintString, options::hint);
}
if (callback != null) {

View File

@@ -30,6 +30,7 @@ import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
import java.util.function.BiFunction;
@@ -110,7 +111,6 @@ import org.springframework.data.mongodb.core.query.NearQuery;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.UpdateDefinition;
import org.springframework.data.mongodb.core.query.UpdateDefinition.ArrayFilter;
import org.springframework.data.mongodb.util.BsonUtils;
import org.springframework.data.projection.EntityProjection;
import org.springframework.data.util.Optionals;
import org.springframework.lang.Nullable;
@@ -938,7 +938,11 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
}
options.getComment().ifPresent(cursor::comment);
options.getHint().ifPresent(cursor::hint);
HintFunction hintFunction = options.getHintObject().map(HintFunction::from).orElseGet(HintFunction::empty);
if (hintFunction.isPresent()) {
cursor = hintFunction.apply(mongoDatabaseFactory, cursor::hintString, cursor::hint);
}
Optionals.firstNonEmpty(options::getCollation, () -> operations.forType(inputType).getCollation()) //
.map(Collation::toMongoCollation) //
@@ -1428,7 +1432,8 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
Document dbDoc = entity.toMappedDocument(writer).getDocument();
maybeEmitEvent(new BeforeSaveEvent<T>(toConvert, dbDoc, collectionName));
return maybeCallBeforeSave(toConvert, dbDoc, collectionName).flatMap(it -> {
return maybeCallBeforeSave(toConvert, dbDoc, collectionName)
.flatMap(it -> {
return saveDocument(collectionName, dbDoc, it.getClass()).flatMap(id -> {
@@ -1441,6 +1446,26 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
});
}
private Mono<Document> resolveValues(Mono<Document> document) {
return document.flatMap(source -> {
for (Entry<String, Object> entry : source.entrySet()) {
if (entry.getValue()instanceof Mono<?> valueMono) {
return valueMono.flatMap(value -> {
source.put(entry.getKey(), value);
return resolveValues(Mono.just(source));
});
}
if (entry.getValue()instanceof Document nested) {
return resolveValues(Mono.just(nested)).map(it -> {
source.put(entry.getKey(), it);
return source;
});
}
}
return Mono.just(source);
});
}
protected Mono<Object> insertDocument(String collectionName, Document dbDoc, Class<?> entityClass) {
if (LOGGER.isDebugEnabled()) {
@@ -1524,15 +1549,16 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
? collection //
: collection.withWriteConcern(writeConcernToUse);
Publisher<?> publisher;
Publisher<?> publisher = null;
Mono<Document> resolved = resolveValues(Mono.just(queryOperations.createInsertContext(mapped).prepareId(entityClass).getDocument()));
if (!mapped.hasId()) {
publisher = collectionToUse.insertOne(queryOperations.createInsertContext(mapped).prepareId(entityClass).getDocument());
publisher = resolved.flatMap(it -> Mono.from(collectionToUse.insertOne(it)));
} else {
MongoPersistentEntity<?> entity = mappingContext.getPersistentEntity(entityClass);
UpdateContext updateContext = queryOperations.replaceSingleContext(mapped, true);
Document filter = updateContext.getMappedQuery(entity);
Document replacement = updateContext.getMappedUpdate(entity);
Mono<Document> replacement = resolveValues(Mono.just(updateContext.getMappedUpdate(entity)));
Mono<Document> deferredFilter;
@@ -1543,14 +1569,17 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
deferredFilter = Mono
.from(
collection.find(filter, Document.class).projection(updateContext.getMappedShardKey(entity)).first())
.defaultIfEmpty(replacement).map(it -> updateContext.applyShardKey(entity, filter, it));
.switchIfEmpty(replacement)
.map(it -> {
return updateContext.applyShardKey(entity, filter, it);
});
}
} else {
deferredFilter = Mono.just(filter);
}
publisher = deferredFilter.flatMapMany(
it -> collectionToUse.replaceOne(it, replacement, updateContext.getReplaceOptions(entityClass)));
publisher = deferredFilter.zipWith(replacement).flatMapMany(
it -> collectionToUse.replaceOne(it.getT1(), it.getT2(), updateContext.getReplaceOptions(entityClass)));
}
return Mono.from(publisher).map(o -> mapped.getId());
@@ -3035,9 +3064,10 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
.map(findPublisher::collation) //
.orElse(findPublisher);
HintFunction hintFunction = HintFunction.from(query.getHint());
Meta meta = query.getMeta();
if (query.getSkip() <= 0 && query.getLimit() <= 0 && ObjectUtils.isEmpty(query.getSortObject())
&& !StringUtils.hasText(query.getHint()) && !meta.hasValues()) {
&& !hintFunction.isPresent() && !meta.hasValues()) {
return findPublisherToUse;
}
@@ -3056,15 +3086,9 @@ public class ReactiveMongoTemplate implements ReactiveMongoOperations, Applicati
findPublisherToUse = findPublisherToUse.sort(sort);
}
if (StringUtils.hasText(query.getHint())) {
String hint = query.getHint();
if (BsonUtils.isJsonDocument(hint)) {
findPublisherToUse = findPublisherToUse.hint(BsonUtils.parse(hint, mongoDatabaseFactory));
} else {
findPublisherToUse = findPublisherToUse.hintString(hint);
}
if (hintFunction.isPresent()) {
findPublisherToUse = hintFunction.apply(mongoDatabaseFactory, findPublisherToUse::hintString,
findPublisherToUse::hint);
}
if (meta.hasValues()) {

View File

@@ -20,6 +20,7 @@ import java.util.Optional;
import org.bson.Document;
import org.springframework.data.mongodb.core.query.Collation;
import org.springframework.data.mongodb.util.BsonUtils;
import org.springframework.lang.Nullable;
import org.springframework.util.Assert;
@@ -33,6 +34,7 @@ import org.springframework.util.Assert;
* @author Christoph Strobl
* @author Mark Paluch
* @author Yadhukrishna S Pai
* @author Soumya Prakash Behera
* @see Aggregation#withOptions(AggregationOptions)
* @see TypedAggregation#withOptions(AggregationOptions)
* @since 1.6
@@ -53,7 +55,7 @@ public class AggregationOptions {
private final Optional<Document> cursor;
private final Optional<Collation> collation;
private final Optional<String> comment;
private final Optional<Document> hint;
private final Optional<Object> hint;
private Duration maxTime = Duration.ZERO;
private ResultOptions resultOptions = ResultOptions.READ;
private DomainTypeMapping domainTypeMapping = DomainTypeMapping.RELAXED;
@@ -65,7 +67,7 @@ public class AggregationOptions {
* @param explain whether to get the execution plan for the aggregation instead of the actual results.
* @param cursor can be {@literal null}, used to pass additional options to the aggregation.
*/
public AggregationOptions(boolean allowDiskUse, boolean explain, Document cursor) {
public AggregationOptions(boolean allowDiskUse, boolean explain, @Nullable Document cursor) {
this(allowDiskUse, explain, cursor, null);
}
@@ -113,7 +115,7 @@ public class AggregationOptions {
* @since 3.1
*/
private AggregationOptions(boolean allowDiskUse, boolean explain, @Nullable Document cursor,
@Nullable Collation collation, @Nullable String comment, @Nullable Document hint) {
@Nullable Collation collation, @Nullable String comment, @Nullable Object hint) {
this.allowDiskUse = allowDiskUse;
this.explain = explain;
@@ -236,12 +238,33 @@ public class AggregationOptions {
}
/**
* Get the hint used to to fulfill the aggregation.
* Get the hint used to fulfill the aggregation.
*
* @return never {@literal null}.
* @since 3.1
* @deprecated since 4.1, use {@link #getHintObject()} instead.
*/
public Optional<Document> getHint() {
return hint.map(it -> {
if (it instanceof Document doc) {
return doc;
}
if (it instanceof String hintString) {
if (BsonUtils.isJsonDocument(hintString)) {
return BsonUtils.parse(hintString, null);
}
}
throw new IllegalStateException("Unable to read hint of type %s".formatted(it.getClass()));
});
}
/**
* Get the hint used to fulfill the aggregation.
*
* @return never {@literal null}.
* @since 4.1
*/
public Optional<Object> getHintObject() {
return hint;
}
@@ -361,7 +384,7 @@ public class AggregationOptions {
private @Nullable Document cursor;
private @Nullable Collation collation;
private @Nullable String comment;
private @Nullable Document hint;
private @Nullable Object hint;
private @Nullable Duration maxTime;
private @Nullable ResultOptions resultOptions;
private @Nullable DomainTypeMapping domainTypeMapping;
@@ -454,6 +477,19 @@ public class AggregationOptions {
return this;
}
/**
* Define a hint that is used by query optimizer to to fulfill the aggregation.
*
* @param indexName can be {@literal null}.
* @return this.
* @since 4.1
*/
public Builder hint(@Nullable String indexName) {
this.hint = indexName;
return this;
}
/**
* Set the time limit for processing.
*

View File

@@ -868,9 +868,9 @@ public class MappingMongoConverter extends AbstractMongoConverter implements App
dbObjectAccessor.put(prop, null);
}
} else if (!conversions.isSimpleType(value.getClass())) {
writePropertyInternal(value, dbObjectAccessor, prop);
writePropertyInternal(value, dbObjectAccessor, prop, accessor);
} else {
writeSimpleInternal(value, bson, prop);
writeSimpleInternal(value, bson, prop, accessor);
}
}
}
@@ -887,11 +887,11 @@ public class MappingMongoConverter extends AbstractMongoConverter implements App
return;
}
writePropertyInternal(value, dbObjectAccessor, inverseProp);
writePropertyInternal(value, dbObjectAccessor, inverseProp, accessor);
}
@SuppressWarnings({ "unchecked" })
protected void writePropertyInternal(@Nullable Object obj, DocumentAccessor accessor, MongoPersistentProperty prop) {
protected void writePropertyInternal(@Nullable Object obj, DocumentAccessor accessor, MongoPersistentProperty prop, PersistentPropertyAccessor<?> persistentPropertyAccessor) {
if (obj == null) {
return;
@@ -902,7 +902,13 @@ public class MappingMongoConverter extends AbstractMongoConverter implements App
if (conversions.hasValueConverter(prop)) {
accessor.put(prop, conversions.getPropertyValueConversions().getValueConverter(prop).write(obj,
new MongoConversionContext(prop, this)));
new MongoConversionContext(new PropertyValueProvider<MongoPersistentProperty>() {
@Nullable
@Override
public <T> T getPropertyValue(MongoPersistentProperty property) {
return (T) persistentPropertyAccessor.getProperty(property);
}
}, prop, this)));
return;
}
@@ -1234,12 +1240,18 @@ public class MappingMongoConverter extends AbstractMongoConverter implements App
BsonUtils.addToMap(bson, key, getPotentiallyConvertedSimpleWrite(value, Object.class));
}
private void writeSimpleInternal(@Nullable Object value, Bson bson, MongoPersistentProperty property) {
private void writeSimpleInternal(@Nullable Object value, Bson bson, MongoPersistentProperty property, PersistentPropertyAccessor<?> persistentPropertyAccessor) {
DocumentAccessor accessor = new DocumentAccessor(bson);
if (conversions.hasValueConverter(property)) {
accessor.put(property, conversions.getPropertyValueConversions().getValueConverter(property).write(value,
new MongoConversionContext(property, this)));
new MongoConversionContext(new PropertyValueProvider<MongoPersistentProperty>() {
@Nullable
@Override
public <T> T getPropertyValue(MongoPersistentProperty property) {
return (T) persistentPropertyAccessor.getProperty(property);
}
}, property, this)));
return;
}
@@ -1892,7 +1904,7 @@ public class MappingMongoConverter extends AbstractMongoConverter implements App
CustomConversions conversions = context.getCustomConversions();
if (conversions.hasValueConverter(property)) {
return (T) conversions.getPropertyValueConversions().getValueConverter(property).read(value,
new MongoConversionContext(property, context.getSourceConverter()));
new MongoConversionContext(this, property, context.getSourceConverter()));
}
ConversionContext contextToUse = context.forProperty(property);

View File

@@ -17,6 +17,8 @@ package org.springframework.data.mongodb.core.convert;
import org.bson.conversions.Bson;
import org.springframework.data.convert.ValueConversionContext;
import org.springframework.data.mapping.PersistentPropertyAccessor;
import org.springframework.data.mapping.model.PropertyValueProvider;
import org.springframework.data.mongodb.core.mapping.MongoPersistentProperty;
import org.springframework.data.util.TypeInformation;
import org.springframework.lang.Nullable;
@@ -29,11 +31,13 @@ import org.springframework.lang.Nullable;
*/
public class MongoConversionContext implements ValueConversionContext<MongoPersistentProperty> {
private final PropertyValueProvider accessor; // TODO: generics
private final MongoPersistentProperty persistentProperty;
private final MongoConverter mongoConverter;
public MongoConversionContext(MongoPersistentProperty persistentProperty, MongoConverter mongoConverter) {
public MongoConversionContext(PropertyValueProvider<?> accessor, MongoPersistentProperty persistentProperty, MongoConverter mongoConverter) {
this.accessor = accessor;
this.persistentProperty = persistentProperty;
this.mongoConverter = mongoConverter;
}
@@ -43,6 +47,10 @@ public class MongoConversionContext implements ValueConversionContext<MongoPersi
return persistentProperty;
}
public Object getValue(String propertyPath) {
return accessor.getPropertyValue(persistentProperty.getOwner().getRequiredPersistentProperty(propertyPath));
}
@Override
public <T> T write(@Nullable Object value, TypeInformation<T> target) {
return (T) mongoConverter.convertToMongoType(value, target);

View File

@@ -437,7 +437,7 @@ public class QueryMapper {
&& converter.getCustomConversions().hasValueConverter(documentField.getProperty())) {
return converter.getCustomConversions().getPropertyValueConversions()
.getValueConverter(documentField.getProperty())
.write(value, new MongoConversionContext(documentField.getProperty(), converter));
.write(value, new MongoConversionContext(null, documentField.getProperty(), converter));
}
if (documentField.isIdField() && !documentField.isAssociation()) {

View File

@@ -17,13 +17,15 @@ package org.springframework.data.mongodb.repository.aot;
import static org.springframework.data.mongodb.aot.MongoAotPredicates.*;
import java.util.Arrays;
import java.util.List;
import org.springframework.aot.hint.MemberCategory;
import org.springframework.aot.hint.RuntimeHints;
import org.springframework.aot.hint.RuntimeHintsRegistrar;
import org.springframework.aot.hint.TypeReference;
import org.springframework.data.querydsl.QuerydslPredicateExecutor;
import org.springframework.data.mongodb.aot.MongoAotPredicates;
import org.springframework.data.mongodb.repository.support.QuerydslMongoPredicateExecutor;
import org.springframework.data.mongodb.repository.support.ReactiveQuerydslMongoPredicateExecutor;
import org.springframework.data.querydsl.QuerydslUtils;
import org.springframework.lang.Nullable;
@@ -37,25 +39,42 @@ class RepositoryRuntimeHints implements RuntimeHintsRegistrar {
public void registerHints(RuntimeHints hints, @Nullable ClassLoader classLoader) {
hints.reflection().registerTypes(
Arrays.asList(TypeReference.of("org.springframework.data.mongodb.repository.support.SimpleMongoRepository")),
List.of(TypeReference.of("org.springframework.data.mongodb.repository.support.SimpleMongoRepository")),
builder -> builder.withMembers(MemberCategory.INVOKE_DECLARED_CONSTRUCTORS,
MemberCategory.INVOKE_PUBLIC_METHODS));
if (isReactorPresent()) {
hints.reflection().registerTypes(
Arrays.asList(
List.of(
TypeReference.of("org.springframework.data.mongodb.repository.support.SimpleReactiveMongoRepository")),
builder -> builder.withMembers(MemberCategory.INVOKE_DECLARED_CONSTRUCTORS,
MemberCategory.INVOKE_PUBLIC_METHODS));
}
if (QuerydslUtils.QUERY_DSL_PRESENT) {
registerQuerydslHints(hints, classLoader);
}
}
hints.reflection().registerType(
TypeReference.of("org.springframework.data.mongodb.repository.support.QuerydslMongoPredicateExecutor"),
hint -> hint.withMembers(MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS, MemberCategory.INVOKE_PUBLIC_METHODS)
.onReachableType(QuerydslPredicateExecutor.class));
/**
* Register hints for Querydsl integration.
*
* @param hints must not be {@literal null}.
* @param classLoader can be {@literal null}.
* @since 4.0.2
*/
private static void registerQuerydslHints(RuntimeHints hints, @Nullable ClassLoader classLoader) {
if (isReactorPresent()) {
hints.reflection().registerType(ReactiveQuerydslMongoPredicateExecutor.class,
MemberCategory.INVOKE_PUBLIC_METHODS, MemberCategory.INVOKE_DECLARED_CONSTRUCTORS);
}
if (MongoAotPredicates.isSyncClientPresent(classLoader)) {
hints.reflection().registerType(QuerydslMongoPredicateExecutor.class, MemberCategory.INVOKE_PUBLIC_METHODS,
MemberCategory.INVOKE_DECLARED_CONSTRUCTORS);
}
}
}

View File

@@ -22,8 +22,6 @@ import org.bson.codecs.DocumentCodec;
import org.bson.json.JsonMode;
import org.bson.json.JsonWriterSettings;
import org.springframework.beans.DirectFieldAccessor;
import com.mongodb.MongoClientSettings;
import com.querydsl.core.support.QueryMixin;
import com.querydsl.core.types.OrderSpecifier;
@@ -49,11 +47,10 @@ abstract class SpringDataMongodbQuerySupport<Q extends SpringDataMongodbQuerySup
@SuppressWarnings("unchecked")
SpringDataMongodbQuerySupport(MongodbDocumentSerializer serializer) {
super(serializer);
this.serializer = serializer;
DirectFieldAccessor fieldAccessor = new DirectFieldAccessor(this);
this.superQueryMixin = (QueryMixin<Q>) fieldAccessor.getPropertyValue("queryMixin");
this.superQueryMixin = super.getQueryMixin();
}
/**

View File

@@ -0,0 +1,105 @@
/*
* Copyright 2022 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.mongodb.classloading;
import java.net.URLClassLoader;
import java.util.Arrays;
import java.util.Collection;
import java.util.stream.Collectors;
import org.springframework.instrument.classloading.ShadowingClassLoader;
import org.springframework.util.Assert;
/**
* is intended for testing code that depends on the presence/absence of certain classes. Classes can be:
* <ul>
* <li>shadowed: reloaded by this classloader no matter if they are loaded already by the SystemClassLoader</li>
* <li>hidden: not loaded by this classloader no matter if they are loaded already by the SystemClassLoader. Trying to
* load these classes results in a {@link ClassNotFoundException}</li>
* <li>all other classes get loaded by the SystemClassLoader</li>
* </ul>
*
* @author Jens Schauder
* @author Oliver Gierke
* @author Christoph Strobl
*/
public class HidingClassLoader extends ShadowingClassLoader {
private final Collection<String> hidden;
public HidingClassLoader(String... hidden) {
this(Arrays.asList(hidden));
}
public HidingClassLoader(Collection<String> hidden) {
super(URLClassLoader.getSystemClassLoader(), false);
this.hidden = hidden;
}
/**
* Creates a new {@link HidingClassLoader} with the packages of the given classes hidden.
*
* @param packages must not be {@literal null}.
* @return
*/
public static HidingClassLoader hide(Class<?>... packages) {
Assert.notNull(packages, "Packages must not be null");
return new HidingClassLoader(Arrays.stream(packages)//
.map(it -> it.getPackage().getName())//
.collect(Collectors.toList()));
}
public static HidingClassLoader hideTypes(Class<?>... types) {
Assert.notNull(types, "Types must not be null!");
return new HidingClassLoader(Arrays.stream(types)//
.map(it -> it.getName())//
.collect(Collectors.toList()));
}
@Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
Class<?> loaded = super.loadClass(name);
checkIfHidden(loaded);
return loaded;
}
@Override
protected boolean isEligibleForShadowing(String className) {
return isExcluded(className);
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
Class<?> loaded = super.findClass(name);
checkIfHidden(loaded);
return loaded;
}
private void checkIfHidden(Class<?> type) throws ClassNotFoundException {
if (hidden.stream().anyMatch(it -> type.getName().startsWith(it))) {
throw new ClassNotFoundException();
}
}
}

View File

@@ -204,6 +204,8 @@ public class MongoTemplateUnitTests extends MongoOperationsUnitTests {
when(aggregateIterable.map(any())).thenReturn(aggregateIterable);
when(aggregateIterable.maxTime(anyLong(), any())).thenReturn(aggregateIterable);
when(aggregateIterable.into(any())).thenReturn(Collections.emptyList());
when(aggregateIterable.hint(any())).thenReturn(aggregateIterable);
when(aggregateIterable.hintString(any())).thenReturn(aggregateIterable);
when(distinctIterable.collation(any())).thenReturn(distinctIterable);
when(distinctIterable.map(any())).thenReturn(distinctIterable);
when(distinctIterable.into(any())).thenReturn(Collections.emptyList());
@@ -497,6 +499,16 @@ public class MongoTemplateUnitTests extends MongoOperationsUnitTests {
verify(aggregateIterable).hint(hint);
}
@Test // GH-4238
void aggregateShouldHonorOptionsHintString() {
AggregationOptions options = AggregationOptions.builder().hint("index-1").build();
template.aggregate(newAggregation(Aggregation.unwind("foo")).withOptions(options), "collection-1", Wrapper.class);
verify(aggregateIterable).hintString("index-1");
}
@Test // GH-3542
void aggregateShouldUseRelaxedMappingByDefault() {

View File

@@ -23,6 +23,8 @@ import static org.springframework.data.mongodb.test.util.Assertions.assertThat;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.springframework.data.mongodb.core.MongoTemplateUnitTests.Wrapper;
import org.springframework.data.mongodb.core.aggregation.Aggregation;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.test.StepVerifier;
@@ -666,6 +668,17 @@ public class ReactiveMongoTemplateUnitTests {
verify(aggregatePublisher).hint(hint);
}
@Test // GH-4238
void aggregateShouldHonorOptionsHintString() {
AggregationOptions options = AggregationOptions.builder().hint("index-1").build();
template.aggregate(newAggregation(Sith.class, project("id")).withOptions(options), AutogenerateableId.class,
Document.class).subscribe();
verify(aggregatePublisher).hintString("index-1");
}
@Test // DATAMONGO-2390
void aggregateShouldNoApplyZeroOrNegativeMaxTime() {

View File

@@ -53,6 +53,7 @@ class AggregationOptionsTests {
assertThat(aggregationOptions.isExplain()).isTrue();
assertThat(aggregationOptions.getCursor()).contains(new Document("batchSize", 1));
assertThat(aggregationOptions.getHint()).contains(dummyHint);
assertThat(aggregationOptions.getHintObject()).contains(dummyHint);
}
@Test // DATAMONGO-1637, DATAMONGO-2153, DATAMONGO-1836
@@ -73,6 +74,7 @@ class AggregationOptionsTests {
assertThat(aggregationOptions.getCursorBatchSize()).isEqualTo(1);
assertThat(aggregationOptions.getComment()).contains("hola");
assertThat(aggregationOptions.getHint()).contains(dummyHint);
assertThat(aggregationOptions.getHintObject()).contains(dummyHint);
}
@Test // DATAMONGO-960, DATAMONGO-2153, DATAMONGO-1836

View File

@@ -2613,7 +2613,7 @@ class MappingMongoConverterUnitTests {
doReturn(Person.class).when(persistentProperty).getType();
doReturn(Person.class).when(persistentProperty).getRawType();
converter.writePropertyInternal(sourceValue, accessor, persistentProperty);
converter.writePropertyInternal(sourceValue, accessor, persistentProperty, null);
assertThat(accessor.getDocument())
.isEqualTo(new org.bson.Document("pName", new org.bson.Document("_id", id.toString())));

View File

@@ -0,0 +1,528 @@
/*
* Copyright 2022 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.mongodb.fle;
import static org.assertj.core.api.Assertions.*;
import static org.springframework.data.mongodb.core.EncryptionAlgorithms.*;
import static org.springframework.data.mongodb.core.query.Criteria.*;
import lombok.Data;
import lombok.Getter;
import lombok.Setter;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import org.bson.BsonArray;
import org.bson.BsonBinary;
import org.bson.BsonDocument;
import org.bson.BsonValue;
import org.bson.Document;
import org.bson.types.Binary;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.CollectionFactory;
import org.springframework.core.annotation.AliasFor;
import org.springframework.dao.PermissionDeniedDataAccessException;
import org.springframework.data.convert.PropertyValueConverterFactory;
import org.springframework.data.convert.ValueConverter;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.convert.MongoConversionContext;
import org.springframework.data.mongodb.core.convert.MongoCustomConversions.MongoConverterConfigurationAdapter;
import org.springframework.data.mongodb.core.convert.MongoValueConverter;
import org.springframework.data.mongodb.core.mapping.Encrypted;
import org.springframework.data.mongodb.core.mapping.MongoPersistentProperty;
import org.springframework.data.mongodb.core.query.Update;
import org.springframework.data.mongodb.fle.FLETests.Config;
import org.springframework.data.mongodb.util.BsonUtils;
import org.springframework.data.util.Lazy;
import org.springframework.lang.Nullable;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.util.ObjectUtils;
import org.springframework.util.StringUtils;
import com.mongodb.ClientEncryptionSettings;
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import com.mongodb.MongoNamespace;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.model.Filters;
import com.mongodb.client.model.IndexOptions;
import com.mongodb.client.model.Indexes;
import com.mongodb.client.model.vault.DataKeyOptions;
import com.mongodb.client.model.vault.EncryptOptions;
import com.mongodb.client.result.DeleteResult;
import com.mongodb.client.vault.ClientEncryption;
import com.mongodb.client.vault.ClientEncryptions;
/**
* @author Christoph Strobl
*/
@ExtendWith(SpringExtension.class)
@ContextConfiguration(classes = Config.class)
public class FLETests {
@Autowired MongoTemplate template;
@Test
void manualEnAndDecryption() {
Person person = new Person();
person.id = "id-1";
person.name = "p1-name";
person.ssn = "mySecretSSN"; // determinisitc encryption (queryable)
person.wallet = "myEvenMoreSecretStuff"; // random encryption (non queryable)
// nested full document encryption
person.address = new Address();
person.address.city = "NYC";
person.address.street = "4th Ave.";
person.encryptedZip = new AddressWithEncryptedZip();
person.encryptedZip.city = "Boston";
person.encryptedZip.street = "central square";
person.encryptedZip.zip = "1234567890";
person.listOfString = Arrays.asList("spring", "data", "mongodb");
Address partOfList = new Address();
partOfList.city = "SFO";
partOfList.street = "---";
person.listOfComplex = Collections.singletonList(partOfList);
template.save(person);
System.out.println("source: " + person);
Document savedDocument = template.execute(Person.class, collection -> {
return collection.find(new Document()).first();
});
// ssn should look like "ssn": {"$binary": {"base64": "...
System.out.println("saved: " + savedDocument.toJson());
assertThat(savedDocument.get("ssn")).isInstanceOf(Binary.class);
assertThat(savedDocument.get("wallet")).isInstanceOf(Binary.class);
assertThat(savedDocument.get("encryptedZip")).isInstanceOf(Document.class);
assertThat(savedDocument.get("encryptedZip", Document.class).get("zip")).isInstanceOf(Binary.class);
assertThat(savedDocument.get("address")).isInstanceOf(Binary.class);
assertThat(savedDocument.get("listOfString")).isInstanceOf(Binary.class);
assertThat(savedDocument.get("listOfComplex")).isInstanceOf(Binary.class);
// count should be 1 using a deterministic algorithm
long queryCount = template.query(Person.class).matching(where("ssn").is(person.ssn)).count();
System.out.println("query(count): " + queryCount);
assertThat(queryCount).isOne();
Person bySsn = template.query(Person.class).matching(where("ssn").is(person.ssn)).firstValue();
System.out.println("queryable: " + bySsn);
assertThat(bySsn).isEqualTo(person);
Person byWallet = template.query(Person.class).matching(where("wallet").is(person.wallet)).firstValue();
System.out.println("not-queryable: " + byWallet);
assertThat(byWallet).isNull();
}
@Test
void theUpdateStuff() {
Person person = new Person();
person.id = "id-1";
person.name = "p1-name";
template.save(person);
Document savedDocument = template.execute(Person.class, collection -> {
return collection.find(new Document()).first();
});
System.out.println("saved: " + savedDocument.toJson());
template.update(Person.class).matching(where("id").is(person.id)).apply(Update.update("ssn", "secret-value")).first();
savedDocument = template.execute(Person.class, collection -> {
return collection.find(new Document()).first();
});
System.out.println("updated: " + savedDocument.toJson());
assertThat(savedDocument.get("ssn")).isInstanceOf(Binary.class);
}
@Test
void altKeyDetection(@Autowired ClientEncryption clientEncryption) throws InterruptedException {
BsonBinary user1key = clientEncryption.createDataKey("local",
new DataKeyOptions().keyAltNames(Collections.singletonList("user-1")));
BsonBinary user2key = clientEncryption.createDataKey("local",
new DataKeyOptions().keyAltNames(Collections.singletonList("user-2")));
Person p1 = new Person();
p1.id = "id-1";
p1.name = "user-1";
p1.ssn = "ssn";
p1.viaAltKeyNameField = "value-1";
Person p2 = new Person();
p2.id = "id-2";
p2.name = "user-2";
p2.viaAltKeyNameField = "value-1";
Person p3 = new Person();
p3.id = "id-3";
p3.name = "user-1";
p3.viaAltKeyNameField = "value-1";
template.save(p1);
template.save(p2);
template.save(p3);
template.execute(Person.class, collection -> {
collection.find(new Document()).forEach(it -> System.out.println(it.toJson()));
return null;
});
// System.out.println(template.query(Person.class).matching(where("id").is(p1.id)).firstValue());
// System.out.println(template.query(Person.class).matching(where("id").is(p2.id)).firstValue());
DeleteResult deleteResult = clientEncryption.deleteKey(user2key);
clientEncryption.getKeys().forEach(System.out::println);
System.out.println("deleteResult: " + deleteResult);
System.out.println("---- waiting for cache timeout ----");
TimeUnit.SECONDS.sleep(90);
assertThat(template.query(Person.class).matching(where("id").is(p1.id)).firstValue()).isEqualTo(p1);
assertThatExceptionOfType(PermissionDeniedDataAccessException.class)
.isThrownBy(() -> template.query(Person.class).matching(where("id").is(p2.id)).firstValue());
}
@Configuration
static class Config extends AbstractMongoClientConfiguration {
@Autowired ApplicationContext applicationContext;
@Override
protected String getDatabaseName() {
return "fle-test";
}
@Bean
public MongoClient mongoClient() {
return super.mongoClient();
}
@Override
protected void configureConverters(MongoConverterConfigurationAdapter converterConfigurationAdapter) {
converterConfigurationAdapter
.registerPropertyValueConverterFactory(PropertyValueConverterFactory.beanFactoryAware(applicationContext));
}
@Bean
EncryptingConverter encryptingConverter(ClientEncryption clientEncryption) {
return new EncryptingConverter(clientEncryption);
}
@Bean
ClientEncryption clientEncryption(MongoClient mongoClient) {
final byte[] localMasterKey = new byte[96];
new SecureRandom().nextBytes(localMasterKey);
Map<String, Map<String, Object>> kmsProviders = new HashMap<String, Map<String, Object>>() {
{
put("local", new HashMap<String, Object>() {
{
put("key", localMasterKey);
}
});
}
};
MongoNamespace keyVaultNamespace = new MongoNamespace("encryption.testKeyVault");
MongoCollection<Document> keyVaultCollection = mongoClient.getDatabase(keyVaultNamespace.getDatabaseName())
.getCollection(keyVaultNamespace.getCollectionName());
keyVaultCollection.drop();
// Ensure that two data keys cannot share the same keyAltName.
keyVaultCollection.createIndex(Indexes.ascending("keyAltNames"),
new IndexOptions().unique(true).partialFilterExpression(Filters.exists("keyAltNames")));
MongoCollection<Document> collection = mongoClient.getDatabase(getDatabaseName()).getCollection("test");
collection.drop(); // Clear old data
// Create the ClientEncryption instance
ClientEncryptionSettings clientEncryptionSettings = ClientEncryptionSettings.builder()
.keyVaultMongoClientSettings(
MongoClientSettings.builder().applyConnectionString(new ConnectionString("mongodb://localhost")).build())
.keyVaultNamespace(keyVaultNamespace.getFullName()).kmsProviders(kmsProviders).build();
ClientEncryption clientEncryption = ClientEncryptions.create(clientEncryptionSettings);
return clientEncryption;
}
}
@Data
@org.springframework.data.mongodb.core.mapping.Document("test")
static class Person {
String id;
String name;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Deterministic) //
String ssn;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Random, altKeyName = "mySuperSecretKey") //
String wallet;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Random) // full document must be random
Address address;
AddressWithEncryptedZip encryptedZip;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Random) // lists must be random
List<String> listOfString;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Random) // lists must be random
List<Address> listOfComplex;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Random, altKeyName = "/name") //
String viaAltKeyNameField;
}
@Data
static class Address {
String city;
String street;
}
@Getter
@Setter
static class AddressWithEncryptedZip extends Address {
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Random) String zip;
@Override
public String toString() {
return "AddressWithEncryptedZip{" + "zip='" + zip + '\'' + ", city='" + getCity() + '\'' + ", street='"
+ getStreet() + '\'' + '}';
}
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Encrypted
@ValueConverter(EncryptingConverter.class)
@interface EncryptedField {
@AliasFor(annotation = Encrypted.class, value = "algorithm")
String algorithm() default "";
String altKeyName() default "";
}
static class EncryptingConverter implements MongoValueConverter<Object, Object> {
private ClientEncryption clientEncryption;
private BsonBinary dataKeyId; // should be provided from outside.
public EncryptingConverter(ClientEncryption clientEncryption) {
this.clientEncryption = clientEncryption;
this.dataKeyId = clientEncryption.createDataKey("local",
new DataKeyOptions().keyAltNames(Collections.singletonList("mySuperSecretKey")));
}
@Nullable
@Override
public Object read(Object value, MongoConversionContext context) {
ManualEncryptionContext encryptionContext = buildEncryptionContext(context);
Object decrypted = encryptionContext.decrypt(value, clientEncryption);
return decrypted instanceof BsonValue ? BsonUtils.toJavaType((BsonValue) decrypted) : decrypted;
}
@Nullable
@Override
public BsonBinary write(Object value, MongoConversionContext context) {
ManualEncryptionContext encryptionContext = buildEncryptionContext(context);
return encryptionContext.encrypt(value, clientEncryption);
}
ManualEncryptionContext buildEncryptionContext(MongoConversionContext context) {
return new ManualEncryptionContext(context, this.dataKeyId);
}
}
static class ManualEncryptionContext {
MongoConversionContext context;
MongoPersistentProperty persistentProperty;
BsonBinary dataKeyId;
Lazy<Encrypted> encryption;
public ManualEncryptionContext(MongoConversionContext context, BsonBinary dataKeyId) {
this.context = context;
this.persistentProperty = context.getProperty();
this.dataKeyId = dataKeyId;
this.encryption = Lazy.of(() -> persistentProperty.findAnnotation(Encrypted.class));
}
BsonBinary encrypt(Object value, ClientEncryption clientEncryption) {
// TODO: check - encryption.get().keyId()
EncryptOptions encryptOptions = new EncryptOptions(encryption.get().algorithm());
EncryptedField annotation = persistentProperty.findAnnotation(EncryptedField.class);
if (annotation != null && !annotation.altKeyName().isBlank()) {
if (annotation.altKeyName().startsWith("/")) {
String fieldName = annotation.altKeyName().replace("/", "");
Object altKeyNameValue = context.getValue(fieldName);
encryptOptions = encryptOptions.keyAltName(altKeyNameValue.toString());
} else {
encryptOptions = encryptOptions.keyAltName(annotation.altKeyName());
}
} else {
encryptOptions = encryptOptions.keyId(this.dataKeyId);
}
System.out.println(
"encrypting with: " + (StringUtils.hasText(encryptOptions.getKeyAltName()) ? encryptOptions.getKeyAltName()
: encryptOptions.getKeyId()));
if (!persistentProperty.isEntity()) {
if (persistentProperty.isCollectionLike()) {
return clientEncryption.encrypt(collectionLikeToBsonValue(value), encryptOptions);
}
return clientEncryption.encrypt(BsonUtils.simpleToBsonValue(value), encryptOptions);
}
if (persistentProperty.isCollectionLike()) {
return clientEncryption.encrypt(collectionLikeToBsonValue(value), encryptOptions);
}
Object write = context.write(value);
if (write instanceof Document doc) {
return clientEncryption.encrypt(doc.toBsonDocument(), encryptOptions);
}
return clientEncryption.encrypt(BsonUtils.simpleToBsonValue(write), encryptOptions);
}
public BsonValue collectionLikeToBsonValue(Object value) {
if (persistentProperty.isCollectionLike()) {
BsonArray bsonArray = new BsonArray();
if (!persistentProperty.isEntity()) {
if (value instanceof Collection values) {
values.forEach(it -> bsonArray.add(BsonUtils.simpleToBsonValue(it)));
} else if (ObjectUtils.isArray(value)) {
for (Object o : ObjectUtils.toObjectArray(value)) {
bsonArray.add(BsonUtils.simpleToBsonValue(o));
}
}
return bsonArray;
} else {
if (value instanceof Collection values) {
values.forEach(it -> {
Document write = (Document) context.write(it, persistentProperty.getTypeInformation());
bsonArray.add(write.toBsonDocument());
});
} else if (ObjectUtils.isArray(value)) {
for (Object o : ObjectUtils.toObjectArray(value)) {
Document write = (Document) context.write(0, persistentProperty.getTypeInformation());
bsonArray.add(write.toBsonDocument());
}
}
return bsonArray;
}
}
if (!persistentProperty.isEntity()) {
if (persistentProperty.isCollectionLike()) {
if (persistentProperty.isEntity()) {
}
}
}
return null;
}
public Object decrypt(Object value, ClientEncryption clientEncryption) {
// this was a hack to avoid the 60 sec timeout of the key cache
// ClientEncryptionSettings settings = (ClientEncryptionSettings) new DirectFieldAccessor(clientEncryption)
// .getPropertyValue("options");
// clientEncryption = ClientEncryptions.create(settings);
Object result = value;
if (value instanceof Binary binary) {
result = clientEncryption.decrypt(new BsonBinary(binary.getType(), binary.getData()));
}
if (value instanceof BsonBinary binary) {
result = clientEncryption.decrypt(binary);
}
// in case the driver has auto decryption (aka .bypassAutoEncryption(true)) active
// https://github.com/mongodb/mongo-java-driver/blob/master/driver-sync/src/examples/tour/ClientSideEncryptionExplicitEncryptionOnlyTour.java
if (value == result) {
return result;
}
if (persistentProperty.isCollectionLike() && result instanceof Iterable<?> iterable) {
if (!persistentProperty.isEntity()) {
Collection<Object> collection = CollectionFactory.createCollection(persistentProperty.getType(), 10);
iterable.forEach(it -> collection.add(BsonUtils.toJavaType((BsonValue) it)));
return collection;
} else {
Collection<Object> collection = CollectionFactory.createCollection(persistentProperty.getType(), 10);
iterable.forEach(it -> {
collection.add(context.read(BsonUtils.toJavaType((BsonValue) it), persistentProperty.getActualType()));
});
return collection;
}
}
if (!persistentProperty.isEntity() && result instanceof BsonValue bsonValue) {
return BsonUtils.toJavaType(bsonValue);
}
if (persistentProperty.isEntity() && result instanceof BsonDocument bsonDocument) {
return context.read(BsonUtils.toJavaType(bsonDocument), persistentProperty.getTypeInformation());
}
return result;
}
}
}

View File

@@ -0,0 +1,440 @@
/*
* Copyright 2022 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.mongodb.fle;
import static org.assertj.core.api.Assertions.*;
import static org.springframework.data.mongodb.core.EncryptionAlgorithms.*;
import static org.springframework.data.mongodb.core.query.Criteria.*;
import lombok.Data;
import org.springframework.data.mongodb.fle.FLETests.Person;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.test.StepVerifier;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.security.SecureRandom;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.concurrent.ExecutionException;
import java.util.function.Supplier;
import org.bson.BsonArray;
import org.bson.BsonBinary;
import org.bson.BsonDocument;
import org.bson.BsonValue;
import org.bson.Document;
import org.bson.types.Binary;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.reactivestreams.Publisher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.CollectionFactory;
import org.springframework.core.annotation.AliasFor;
import org.springframework.data.convert.PropertyValueConverterFactory;
import org.springframework.data.convert.ValueConverter;
import org.springframework.data.mongodb.config.AbstractReactiveMongoConfiguration;
import org.springframework.data.mongodb.core.ReactiveMongoTemplate;
import org.springframework.data.mongodb.core.convert.MongoConversionContext;
import org.springframework.data.mongodb.core.convert.MongoCustomConversions.MongoConverterConfigurationAdapter;
import org.springframework.data.mongodb.core.convert.MongoValueConverter;
import org.springframework.data.mongodb.core.mapping.Encrypted;
import org.springframework.data.mongodb.core.mapping.MongoPersistentProperty;
import org.springframework.data.mongodb.fle.FLETests.Config;
import org.springframework.data.mongodb.util.BsonUtils;
import org.springframework.data.util.Lazy;
import org.springframework.lang.Nullable;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.util.ObjectUtils;
import org.springframework.util.StringUtils;
import com.mongodb.ClientEncryptionSettings;
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import com.mongodb.MongoNamespace;
import com.mongodb.client.model.Filters;
import com.mongodb.client.model.IndexOptions;
import com.mongodb.client.model.Indexes;
import com.mongodb.client.model.vault.DataKeyOptions;
import com.mongodb.client.model.vault.EncryptOptions;
import com.mongodb.reactivestreams.client.MongoClient;
import com.mongodb.reactivestreams.client.MongoCollection;
import com.mongodb.reactivestreams.client.vault.ClientEncryption;
import com.mongodb.reactivestreams.client.vault.ClientEncryptions;
/**
* @author Christoph Strobl
* @since 2022/11
*/
@ExtendWith(SpringExtension.class)
@ContextConfiguration(classes = ReactiveFLETests.Config.class)
public class ReactiveFLETests {
ClientEncryption encryption;
@Test
void xxx() {
Supplier<String> valueSupplier = new Supplier<String>() {
@Override
public String get() {
System.out.println("invoked");
return "v1";
}
};
Document source = new Document("name", "value").append("mono", Mono.fromSupplier(() -> "from mono"))
.append("nested", new Document("n1", Mono.fromSupplier(() -> "from nested mono")));
resolveValues(Mono.just(source)) //
.as(StepVerifier::create).consumeNextWith(resolved -> {
assertThat(resolved).isEqualTo(Document
.parse("{\"name\": \"value\", \"mono\": \"from mono\", \"nested\" : { \"n1\" : \"from nested mono\"}}"));
}).verifyComplete();
}
private Mono<Document> resolveValues(Mono<Document> document) {
return document.flatMap(source -> {
for (Entry<String, Object> entry : source.entrySet()) {
if (entry.getValue()instanceof Mono<?> valueMono) {
return valueMono.flatMap(value -> {
source.put(entry.getKey(), value);
return resolveValues(Mono.just(source));
});
}
if (entry.getValue()instanceof Document nested) {
return resolveValues(Mono.just(nested)).map(it -> {
source.put(entry.getKey(), it);
return source;
});
}
}
return Mono.just(source);
});
}
@Autowired ReactiveMongoTemplate template;
@Test
void manualEnAndDecryption() {
Person person = new Person();
person.id = "id-1";
person.name = "p1-name";
person.ssn = "mySecretSSN";
template.save(person).block();
System.out.println("source: " + person);
Flux<Document> result = template.execute(FLETests.Person.class, collection -> {
return Mono.from(collection.find(new Document()).first());
});
System.out.println("encrypted: " + result.blockFirst().toJson());
Person id = template.query(Person.class).matching(where("id").is(person.id)).first().block();
System.out.println("decrypted: " + id);
}
@Data
@org.springframework.data.mongodb.core.mapping.Document("test")
static class Person {
String id;
String name;
@EncryptedField(algorithm = AEAD_AES_256_CBC_HMAC_SHA_512_Deterministic) //
String ssn;
}
@Configuration
static class Config extends AbstractReactiveMongoConfiguration {
@Autowired ApplicationContext applicationContext;
@Override
protected String getDatabaseName() {
return "fle-test";
}
@Override
protected void configureConverters(MongoConverterConfigurationAdapter converterConfigurationAdapter) {
converterConfigurationAdapter
.registerPropertyValueConverterFactory(PropertyValueConverterFactory.beanFactoryAware(applicationContext));
}
@Bean
@Override
public MongoClient reactiveMongoClient() {
return super.reactiveMongoClient();
}
@Bean
ReactiveEncryptingConverter encryptingConverter(ClientEncryption clientEncryption) {
return new ReactiveEncryptingConverter(clientEncryption);
}
@Bean
ClientEncryption clientEncryption(MongoClient mongoClient) {
final byte[] localMasterKey = new byte[96];
new SecureRandom().nextBytes(localMasterKey);
Map<String, Map<String, Object>> kmsProviders = new HashMap<String, Map<String, Object>>() {
{
put("local", new HashMap<String, Object>() {
{
put("key", localMasterKey);
}
});
}
};
MongoNamespace keyVaultNamespace = new MongoNamespace("encryption.testKeyVault");
MongoCollection<Document> keyVaultCollection = mongoClient.getDatabase(keyVaultNamespace.getDatabaseName())
.getCollection(keyVaultNamespace.getCollectionName());
Mono.from(keyVaultCollection.drop()).block();
// Ensure that two data keys cannot share the same keyAltName.
keyVaultCollection.createIndex(Indexes.ascending("keyAltNames"),
new IndexOptions().unique(true).partialFilterExpression(Filters.exists("keyAltNames")));
MongoCollection<Document> collection = mongoClient.getDatabase(getDatabaseName()).getCollection("test");
Mono.from(collection.drop()).block(); // Clear old data
// Create the ClientEncryption instance
ClientEncryptionSettings clientEncryptionSettings = ClientEncryptionSettings.builder()
.keyVaultMongoClientSettings(
MongoClientSettings.builder().applyConnectionString(new ConnectionString("mongodb://localhost")).build())
.keyVaultNamespace(keyVaultNamespace.getFullName()).kmsProviders(kmsProviders).build();
ClientEncryption clientEncryption = ClientEncryptions.create(clientEncryptionSettings);
return clientEncryption;
}
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Encrypted
@ValueConverter(ReactiveEncryptingConverter.class)
@interface EncryptedField {
@AliasFor(annotation = Encrypted.class, value = "algorithm")
String algorithm() default "";
String altKeyName() default "";
}
static class ReactiveEncryptingConverter implements MongoValueConverter<Object, Object> {
private ClientEncryption clientEncryption;
private BsonBinary dataKeyId; // should be provided from outside.
public ReactiveEncryptingConverter(ClientEncryption clientEncryption) {
this.clientEncryption = clientEncryption;
this.dataKeyId = Mono.from(clientEncryption.createDataKey("local",
new DataKeyOptions().keyAltNames(Collections.singletonList("mySuperSecretKey")))).block();
}
@Nullable
@Override
public Object read(Object value, MongoConversionContext context) {
ManualEncryptionContext encryptionContext = buildEncryptionContext(context);
Object decrypted = null;
try {
decrypted = encryptionContext.decrypt(value, clientEncryption);
} catch (ExecutionException e) {
throw new RuntimeException(e);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return decrypted instanceof BsonValue ? BsonUtils.toJavaType((BsonValue) decrypted) : decrypted;
}
@Nullable
@Override
public Publisher<BsonBinary> write(Object value, MongoConversionContext context) {
ManualEncryptionContext encryptionContext = buildEncryptionContext(context);
return encryptionContext.encrypt(value, clientEncryption);
}
ManualEncryptionContext buildEncryptionContext(MongoConversionContext context) {
return new ManualEncryptionContext(context, this.dataKeyId);
}
}
static class ManualEncryptionContext {
MongoConversionContext context;
MongoPersistentProperty persistentProperty;
BsonBinary dataKeyId;
Lazy<Encrypted> encryption;
public ManualEncryptionContext(MongoConversionContext context, BsonBinary dataKeyId) {
this.context = context;
this.persistentProperty = context.getProperty();
this.dataKeyId = dataKeyId;
this.encryption = Lazy.of(() -> persistentProperty.findAnnotation(Encrypted.class));
}
Publisher<BsonBinary> encrypt(Object value, ClientEncryption clientEncryption) {
// TODO: check - encryption.get().keyId()
EncryptOptions encryptOptions = new EncryptOptions(encryption.get().algorithm());
EncryptedField annotation = persistentProperty.findAnnotation(EncryptedField.class);
if (annotation != null && !annotation.altKeyName().isBlank()) {
if (annotation.altKeyName().startsWith("/")) {
String fieldName = annotation.altKeyName().replace("/", "");
Object altKeyNameValue = context.getValue(fieldName);
encryptOptions = encryptOptions.keyAltName(altKeyNameValue.toString());
} else {
encryptOptions = encryptOptions.keyAltName(annotation.altKeyName());
}
} else {
encryptOptions = encryptOptions.keyId(this.dataKeyId);
}
System.out.println(
"encrypting with: " + (StringUtils.hasText(encryptOptions.getKeyAltName()) ? encryptOptions.getKeyAltName()
: encryptOptions.getKeyId()));
if (!persistentProperty.isEntity()) {
if (persistentProperty.isCollectionLike()) {
return clientEncryption.encrypt(collectionLikeToBsonValue(value), encryptOptions);
}
return clientEncryption.encrypt(BsonUtils.simpleToBsonValue(value), encryptOptions);
}
if (persistentProperty.isCollectionLike()) {
return clientEncryption.encrypt(collectionLikeToBsonValue(value), encryptOptions);
}
Object write = context.write(value);
if (write instanceof Document doc) {
return clientEncryption.encrypt(doc.toBsonDocument(), encryptOptions);
}
return clientEncryption.encrypt(BsonUtils.simpleToBsonValue(write), encryptOptions);
}
public BsonValue collectionLikeToBsonValue(Object value) {
if (persistentProperty.isCollectionLike()) {
BsonArray bsonArray = new BsonArray();
if (!persistentProperty.isEntity()) {
if (value instanceof Collection values) {
values.forEach(it -> bsonArray.add(BsonUtils.simpleToBsonValue(it)));
} else if (ObjectUtils.isArray(value)) {
for (Object o : ObjectUtils.toObjectArray(value)) {
bsonArray.add(BsonUtils.simpleToBsonValue(o));
}
}
return bsonArray;
} else {
if (value instanceof Collection values) {
values.forEach(it -> {
Document write = (Document) context.write(it, persistentProperty.getTypeInformation());
bsonArray.add(write.toBsonDocument());
});
} else if (ObjectUtils.isArray(value)) {
for (Object o : ObjectUtils.toObjectArray(value)) {
Document write = (Document) context.write(o, persistentProperty.getTypeInformation());
bsonArray.add(write.toBsonDocument());
}
}
return bsonArray;
}
}
if (!persistentProperty.isEntity()) {
if (persistentProperty.isCollectionLike()) {
if (persistentProperty.isEntity()) {
}
}
}
return null;
}
public Object decrypt(Object value, ClientEncryption clientEncryption) throws ExecutionException, InterruptedException {
// this was a hack to avoid the 60 sec timeout of the key cache
// ClientEncryptionSettings settings = (ClientEncryptionSettings) new DirectFieldAccessor(clientEncryption)
// .getPropertyValue("options");
// clientEncryption = ClientEncryptions.create(settings);
Object r = value;
if (value instanceof Binary binary) {
r = clientEncryption.decrypt(new BsonBinary(binary.getType(), binary.getData()));
}
if (value instanceof BsonBinary binary) {
r = clientEncryption.decrypt(binary);
}
// in case the driver has auto decryption (aka .bypassAutoEncryption(true)) active
// https://github.com/mongodb/mongo-java-driver/blob/master/driver-sync/src/examples/tour/ClientSideEncryptionExplicitEncryptionOnlyTour.java
if (value == r) {
return r;
}
if(r instanceof Mono mono) {
return mono.map(result -> {
if (persistentProperty.isCollectionLike() && result instanceof Iterable<?> iterable) {
if (!persistentProperty.isEntity()) {
Collection<Object> collection = CollectionFactory.createCollection(persistentProperty.getType(), 10);
iterable.forEach(it -> collection.add(BsonUtils.toJavaType((BsonValue) it)));
return collection;
} else {
Collection<Object> collection = CollectionFactory.createCollection(persistentProperty.getType(), 10);
iterable.forEach(it -> {
collection.add(context.read(BsonUtils.toJavaType((BsonValue) it), persistentProperty.getActualType()));
});
return collection;
}
}
if (!persistentProperty.isEntity() && result instanceof BsonValue bsonValue) {
return BsonUtils.toJavaType(bsonValue);
}
if (persistentProperty.isEntity() && result instanceof BsonDocument bsonDocument) {
return context.read(BsonUtils.toJavaType(bsonDocument), persistentProperty.getTypeInformation());
}
return result;
}).toFuture().get();
}
return r;
}
}
}

View File

@@ -0,0 +1,67 @@
/*
* Copyright 2023 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.mongodb.repository.aot;
import static org.assertj.core.api.Assertions.*;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.springframework.aot.hint.RuntimeHints;
import org.springframework.aot.hint.predicate.RuntimeHintsPredicates;
import org.springframework.data.mongodb.classloading.HidingClassLoader;
import org.springframework.data.mongodb.repository.support.QuerydslMongoPredicateExecutor;
import org.springframework.data.mongodb.repository.support.ReactiveQuerydslMongoPredicateExecutor;
import com.mongodb.client.MongoClient;
/**
* Unit tests for {@link RepositoryRuntimeHints}.
*
* @author Christoph Strobl
*/
class RepositoryRuntimeHintsUnitTests {
@Test // GH-4244
void registersTypesForQuerydslIntegration() {
RuntimeHints runtimeHints = new RuntimeHints();
new RepositoryRuntimeHints().registerHints(runtimeHints, null);
assertThat(runtimeHints).matches(RuntimeHintsPredicates.reflection().onType(QuerydslMongoPredicateExecutor.class)
.and(RuntimeHintsPredicates.reflection().onType(ReactiveQuerydslMongoPredicateExecutor.class)));
}
@Test // GH-4244
void onlyRegistersReactiveTypesForQuerydslIntegrationWhenNoSyncClientPresent() {
RuntimeHints runtimeHints = new RuntimeHints();
new RepositoryRuntimeHints().registerHints(runtimeHints, HidingClassLoader.hide(MongoClient.class));
assertThat(runtimeHints).matches(RuntimeHintsPredicates.reflection().onType(QuerydslMongoPredicateExecutor.class)
.negate().and(RuntimeHintsPredicates.reflection().onType(ReactiveQuerydslMongoPredicateExecutor.class)));
}
@Test // GH-4244
@Disabled("TODO: ReactiveWrappers does not support ClassLoader")
void doesNotRegistersReactiveTypesForQuerydslIntegrationWhenReactorNotPresent() {
RuntimeHints runtimeHints = new RuntimeHints();
new RepositoryRuntimeHints().registerHints(runtimeHints, new HidingClassLoader("reactor.core"));
assertThat(runtimeHints).matches(RuntimeHintsPredicates.reflection().onType(QuerydslMongoPredicateExecutor.class)
.and(RuntimeHintsPredicates.reflection().onType(ReactiveQuerydslMongoPredicateExecutor.class).negate()));
}
}

View File

@@ -10,13 +10,13 @@ This section provides some basic introduction to Spring and Document databases.
[[get-started:first-steps:spring]]
== Learning Spring
Spring Data uses Spring framework's https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html[core] functionality, including:
Spring Data uses Spring framework's link:{springDocsUrl}/core.html[core] functionality, including:
* https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html#beans[IoC] container
* https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html#validation[type conversion system]
* https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html#expressions[expression language]
* https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/integration.html#jmx[JMX integration]
* https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/data-access.html#dao-exceptions[DAO exception hierarchy].
* link:{springDocsUrl}/core.html#beans[IoC] container
* link:{springDocsUrl}/core.html#validation[type conversion system]
* link:{springDocsUrl}/core.html#expressions[expression language]
* link:{springDocsUrl}/integration.html#jmx[JMX integration]
* link:{springDocsUrl}/data-access.html#dao-exceptions[DAO exception hierarchy].
While you need not know the Spring APIs, understanding the concepts behind them is important. At a minimum, the idea behind Inversion of Control (IoC) should be familiar, and you should be familiar with whatever IoC container you choose to use.

View File

@@ -177,7 +177,7 @@ CAUTION: Changing state of `MongoTemplate` during runtime (as you might think wo
[[mongo.transactions.tx-manager]]
== Transactions with `MongoTransactionManager`
`MongoTransactionManager` is the gateway to the well known Spring transaction support. It lets applications use https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/data-access.html#transaction[the managed transaction features of Spring].
`MongoTransactionManager` is the gateway to the well known Spring transaction support. It lets applications use link:{springDocsUrl}/data-access.html#transaction[the managed transaction features of Spring].
The `MongoTransactionManager` binds a `ClientSession` to the thread. `MongoTemplate` detects the session and operates on these resources which are associated with the transaction accordingly. `MongoTemplate` can also participate in other, ongoing transactions. The following example shows how to create and use transactions with a `MongoTransactionManager`:
.Transactions with `MongoTransactionManager`

View File

@@ -437,7 +437,7 @@ class Entity {
// referenced object
{
"_id" : "9a48e32",
"firsntame" : "Josh", <2>
"firstname" : "Josh", <2>
"lastname" : "Long", <2>
}
----

View File

@@ -1,7 +1,7 @@
[[mongo.jmx]]
= JMX support
The JMX support for MongoDB exposes the results of running the 'serverStatus' command on the admin database for a single MongoDB server instance. It also exposes an administrative MBean, `MongoAdmin`, that lets you perform administrative operations, such as dropping or creating a database. The JMX features build upon the JMX feature set available in the Spring Framework. See https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/integration.html#jmx[here] for more details.
The JMX support for MongoDB exposes the results of running the 'serverStatus' command on the admin database for a single MongoDB server instance. It also exposes an administrative MBean, `MongoAdmin`, that lets you perform administrative operations, such as dropping or creating a database. The JMX features build upon the JMX feature set available in the Spring Framework. See link:{springDocsUrl}/integration.html#jmx[here] for more details.
[[mongodb:jmx-configuration]]
== MongoDB JMX Configuration

View File

@@ -40,7 +40,7 @@ The snippet above is handy for providing simple type hints. To gain more fine-gr
The `MappingMongoConverter` checks to see if any Spring converters can handle a specific class before attempting to map the object itself. To 'hijack' the normal mapping strategies of the `MappingMongoConverter`, perhaps for increased performance or other custom mapping needs, you first need to create an implementation of the Spring `Converter` interface and then register it with the `MappingConverter`.
NOTE: For more information on the Spring type conversion service, see the reference docs https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html#validation[here].
NOTE: For more information on the Spring type conversion service, see the reference docs link:{springDocsUrl}/core.html#validation[here].
[[mongo.custom-converters.writer]]
=== Saving by Using a Registered Spring Converter

View File

@@ -3,13 +3,13 @@
The repository layer offers means to interact with <<mongo.aggregation, the aggregation framework>> via annotated repository query methods.
Similar to the <<mongodb.repositories.queries.json-based, JSON based queries>>, you can define a pipeline using the `org.springframework.data.mongodb.repository.Aggregation` annotation.
The definition may contain simple placeholders like `?0` as well as https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html#expressions[SpEL expressions] `?#{ … }`.
The definition may contain simple placeholders like `?0` as well as link:{springDocsUrl}/core.html#expressions[SpEL expressions] `?#{ … }`.
.Aggregating Repository Method
====
[source,java]
----
public interface PersonRepository extends CrudReppsitory<Person, String> {
public interface PersonRepository extends CrudRepository<Person, String> {
@Aggregation("{ $group: { _id : $lastname, names : { $addToSet : $firstname } } }")
List<PersonAggregate> groupByLastnameAndFirstnames(); <1>
@@ -82,7 +82,7 @@ Use the `@Meta` annotation to set those options via `maxExecutionTimeMs`, `comme
[source,java]
----
interface PersonRepository extends CrudReppsitory<Person, String> {
interface PersonRepository extends CrudRepository<Person, String> {
@Meta(allowDiskUse = true)
@Aggregation("{ $group: { _id : $lastname, names : { $addToSet : $firstname } } }")
@@ -99,7 +99,7 @@ Or use `@Meta` to create your own annotation as shown in the sample below.
@Meta(allowDiskUse = true)
@interface AllowDiskUse { }
interface PersonRepository extends CrudReppsitory<Person, String> {
interface PersonRepository extends CrudRepository<Person, String> {
@AllowDiskUse
@Aggregation("{ $group: { _id : $lastname, names : { $addToSet : $firstname } } }")

View File

@@ -191,7 +191,7 @@ public class AppConfig {
----
====
This approach lets you use the standard `com.mongodb.client.MongoClient` instance, with the container using Spring's `MongoClientFactoryBean`. As compared to instantiating a `com.mongodb.client.MongoClient` instance directly, the `FactoryBean` has the added advantage of also providing the container with an `ExceptionTranslator` implementation that translates MongoDB exceptions to exceptions in Spring's portable `DataAccessException` hierarchy for data access classes annotated with the `@Repository` annotation. This hierarchy and the use of `@Repository` is described in https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/data-access.html[Spring's DAO support features].
This approach lets you use the standard `com.mongodb.client.MongoClient` instance, with the container using Spring's `MongoClientFactoryBean`. As compared to instantiating a `com.mongodb.client.MongoClient` instance directly, the `FactoryBean` has the added advantage of also providing the container with an `ExceptionTranslator` implementation that translates MongoDB exceptions to exceptions in Spring's portable `DataAccessException` hierarchy for data access classes annotated with the `@Repository` annotation. This hierarchy and the use of `@Repository` is described in link:{springDocsUrl}/data-access.html[Spring's DAO support features].
The following example shows an example of a Java-based bean metadata that supports exception translation on `@Repository` annotated classes:
@@ -2198,7 +2198,7 @@ directly there are several methods for those options.
----
Query query = query(where("firstname").is("luke"))
.comment("find luke") <1>
.batchSize(100) <2>
.cursorBatchSize(100) <2>
----
<1> The comment propagated to the MongoDB profile log.
<2> The number of documents to return in each response batch.
@@ -2209,7 +2209,7 @@ On the repository level the `@Meta` annotation provides means to add query optio
====
[source,java]
----
@Meta(comment = "find luke", batchSize = 100, flags = { SLAVE_OK })
@Meta(comment = "find luke", cursorBatchSize = 100, flags = { SLAVE_OK })
List<Person> findByFirstname(String firstname);
----
====
@@ -2258,7 +2258,7 @@ Therefore a given `Query` will be rewritten for `count` operations using `Reacti
You can query MongoDB by using Map-Reduce, which is useful for batch processing, for data aggregation, and for when the query language does not fulfill your needs.
Spring provides integration with MongoDB's Map-Reduce by providing methods on `MongoOperations` to simplify the creation and running of Map-Reduce operations.It can convert the results of a Map-Reduce operation to a POJO and integrates with Spring's https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/core.html#resources[Resource abstraction].This lets you place your JavaScript files on the file system, classpath, HTTP server, or any other Spring Resource implementation and then reference the JavaScript resources through an easy URI style syntax -- for example, `classpath:reduce.js;`.Externalizing JavaScript code in files is often preferable to embedding them as Java strings in your code.Note that you can still pass JavaScript code as Java strings if you prefer.
Spring provides integration with MongoDB's Map-Reduce by providing methods on `MongoOperations` to simplify the creation and running of Map-Reduce operations.It can convert the results of a Map-Reduce operation to a POJO and integrates with Spring's link:{springDocsUrl}/core.html#resources[Resource abstraction].This lets you place your JavaScript files on the file system, classpath, HTTP server, or any other Spring Resource implementation and then reference the JavaScript resources through an easy URI style syntax -- for example, `classpath:reduce.js;`.Externalizing JavaScript code in files is often preferable to embedding them as Java strings in your code.Note that you can still pass JavaScript code as Java strings if you prefer.
[[mongo.mapreduce.example]]
=== Example Usage
@@ -2664,7 +2664,7 @@ include::./mongo-entity-callbacks.adoc[leveloffset=+2]
The Spring framework provides exception translation for a wide variety of database and mapping technologies. This has traditionally been for JDBC and JPA. The Spring support for MongoDB extends this feature to the MongoDB Database by providing an implementation of the `org.springframework.dao.support.PersistenceExceptionTranslator` interface.
The motivation behind mapping to Spring's https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/data-access.html#dao-exceptions[consistent data access exception hierarchy] is that you are then able to write portable and descriptive exception handling code without resorting to coding against MongoDB error codes. All of Spring's data access exceptions are inherited from the root `DataAccessException` class so that you can be sure to catch all database related exception within a single try-catch block. Note that not all exceptions thrown by the MongoDB driver inherit from the `MongoException` class. The inner exception and message are preserved so that no information is lost.
The motivation behind mapping to Spring's link:{springDocsUrl}/data-access.html#dao-exceptions[consistent data access exception hierarchy] is that you are then able to write portable and descriptive exception handling code without resorting to coding against MongoDB error codes. All of Spring's data access exceptions are inherited from the root `DataAccessException` class so that you can be sure to catch all database related exception within a single try-catch block. Note that not all exceptions thrown by the MongoDB driver inherit from the `MongoException` class. The inner exception and message are preserved so that no information is lost.
Some of the mappings performed by the `MongoExceptionTranslator` are `com.mongodb.Network to DataAccessResourceFailureException` and `MongoException` error codes 1003, 12001, 12010, 12011, and 12012 to `InvalidDataAccessApiUsageException`. Look into the implementation for more details on the mapping.

View File

@@ -164,7 +164,7 @@ public class AppConfig {
This approach lets you use the standard `com.mongodb.reactivestreams.client.MongoClient` API (which you may already know).
An alternative is to register an instance of `com.mongodb.reactivestreams.client.MongoClient` instance with the container by using Spring's `ReactiveMongoClientFactoryBean`. As compared to instantiating a `com.mongodb.reactivestreams.client.MongoClient` instance directly, the `FactoryBean` approach has the added advantage of also providing the container with an `ExceptionTranslator` implementation that translates MongoDB exceptions to exceptions in Spring's portable `DataAccessException` hierarchy for data access classes annotated with the `@Repository` annotation. This hierarchy and use of `@Repository` is described in https://docs.spring.io/spring-framework/docs/{springVersion}/reference/html/data-access.html[Spring's DAO support features].
An alternative is to register an instance of `com.mongodb.reactivestreams.client.MongoClient` instance with the container by using Spring's `ReactiveMongoClientFactoryBean`. As compared to instantiating a `com.mongodb.reactivestreams.client.MongoClient` instance directly, the `FactoryBean` approach has the added advantage of also providing the container with an `ExceptionTranslator` implementation that translates MongoDB exceptions to exceptions in Spring's portable `DataAccessException` hierarchy for data access classes annotated with the `@Repository` annotation. This hierarchy and use of `@Repository` is described in link:{springDocsUrl}/data-access.html[Spring's DAO support features].
The following example shows Java-based bean metadata that supports exception translation on `@Repository` annotated classes:

View File

@@ -1,4 +1,4 @@
Spring Data MongoDB 4.0.1 (2022.0.1)
Spring Data MongoDB 4.0 GA (2022.0.0)
Copyright (c) [2010-2019] Pivotal Software, Inc.
This product is licensed to you under the Apache License, Version 2.0 (the "License").
@@ -40,6 +40,5 @@ conditions of the subcomponent's license, as noted in the LICENSE file.