Could not save date field as ISO date in mongo db

2019-02-21 00:31发布

问题:

I have pojo like this:

@Document(collection = "data")
public class DataPoint {
    @Id
    private String id;
    private LocalDateTime createdDate;
    ....
}

in some code base I have following code:

@Autowired
private ProducerTemplate producerTemplate;

...

final List<DataPoint> dataPoints =....
producerTemplate.sendBody("mongodb:mongoBean?database=" + mongoDataConfiguration.getDatabase()
                                    + "&createCollection=true&operation=insert&collection=" + mongoDataConfiguration.getDataPointCollection(), 
        dataPoints);

But when I open collection in database I see date field like this:

"createdDate" : {
                "month" : "NOVEMBER",
                "year" : 2017,
                "dayOfMonth" : 7,
                "dayOfWeek" : "TUESDAY",
                "dayOfYear" : 311,
                "monthValue" : 11,
                "hour" : 17,
                "minute" : 55,
                "nano" : 259000000,
                "second" : 21,
                "chronology" : {
                        "id" : "ISO",
                        "calendarType" : "iso8601"
                }

But I want to store usual ISO date recognizable for mongoDB. It should be something like this:

ISODate("2017-11-06T12:47:51.720")

How did I try to resolve this issue?

1:
I read this topic:

Thus I created a custom serializer:

public class IsoDateSerializer extends JsonSerializer<DateTime> {
    @Override
    public void serialize(DateTime value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
           jgen.writeStartObject();
        serializeContents(value.toDate(), jgen, provider);
        jgen.writeEndObject();
    }
    private void serializeContents(Date value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
        jgen.writeFieldName("$date");
        SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
        String formattedDate = formatter.format(value);
        jgen.writeString(formattedDate);
    }
}

and registered it correctly:

@JsonSerialize(using = IsoDateSerializer.class)
public DateTime getCreatedDate() {
    return new DateTime(Date.from(createdDate.toInstant(ZoneOffset.UTC)));
}

But it throws exception:

org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-host-1510066910268-0-3]
        at org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1847) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.util.ExchangeHelper.extractResultBody(ExchangeHelper.java:713) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.DefaultProducerTemplate.extractResultBody(DefaultProducerTemplate.java:515) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.DefaultProducerTemplate.extractResultBody(DefaultProducerTemplate.java:511) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:163) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:168) ~[camel-core-2.20.0.jar:2.20.0]
        at com.debeers.mis.upload.route.DummyRoute$1.process(DummyRoute.java:113) ~[classes/:na]
        at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:219) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:183) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174) [camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101) [camel-core-2.20.0.jar:2.20.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_111]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_111]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_111]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_111]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
    Caused by: org.apache.camel.component.mongodb.CamelMongoDbException: java.lang.IllegalArgumentException: Invalid BSON field name $date
        at org.apache.camel.component.mongodb.MongoDbComponent.wrapInCamelMongoDbException(MongoDbComponent.java:61) ~[camel-mongodb-2.20.0.jar:2.20.0]
        at org.apache.camel.component.mongodb.MongoDbProducer.process(MongoDbProducer.java:98) ~[camel-mongodb-2.20.0.jar:2.20.0]
        at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:186) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:86) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:541) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:506) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:369) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:506) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:229) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:144) ~[camel-core-2.20.0.jar:2.20.0]
        at org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:161) ~[camel-core-2.20.0.jar:2.20.0]
        ... 18 common frames omitted
    Caused by: java.lang.IllegalArgumentException: Invalid BSON field name $date
        at org.bson.AbstractBsonWriter.writeName(AbstractBsonWriter.java:516) ~[bson-3.4.3.jar:na]
        at com.mongodb.DBObjectCodec.encodeMap(DBObjectCodec.java:221) ~[mongodb-driver-3.4.3.jar:na]
        at com.mongodb.DBObjectCodec.writeValue(DBObjectCodec.java:198) ~[mongodb-driver-3.4.3.jar:na]
        at com.mongodb.DBObjectCodec.encode(DBObjectCodec.java:130) ~[mongodb-driver-3.4.3.jar:na]
        at com.mongodb.DBObjectCodec.encode(DBObjectCodec.java:61) ~[mongodb-driver-3.4.3.jar:na]
        at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63) ~[bson-3.4.3.jar:na]
        at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29) ~[bson-3.4.3.jar:na]
        at com.mongodb.connection.InsertCommandMessage.writeTheWrites(InsertCommandMessage.java:101) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.InsertCommandMessage.writeTheWrites(InsertCommandMessage.java:43) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.BaseWriteCommandMessage.encodeMessageBodyWithMetadata(BaseWriteCommandMessage.java:129) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.RequestMessage.encodeWithMetadata(RequestMessage.java:160) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.WriteCommandProtocol.sendMessage(WriteCommandProtocol.java:220) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.WriteCommandProtocol.execute(WriteCommandProtocol.java:101) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.InsertCommandProtocol.execute(InsertCommandProtocol.java:67) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.InsertCommandProtocol.execute(InsertCommandProtocol.java:37) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.connection.DefaultServerConnection.insertCommand(DefaultServerConnection.java:118) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation$Run$2.executeWriteCommandProtocol(MixedBulkWriteOperation.java:465) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation$Run$RunExecutor.execute(MixedBulkWriteOperation.java:656) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation$Run.execute(MixedBulkWriteOperation.java:411) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:177) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:168) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:426) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:417) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:168) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:74) ~[mongodb-driver-core-3.4.3.jar:na]
        at com.mongodb.Mongo.execute(Mongo.java:845) ~[mongodb-driver-3.4.3.jar:na]
        at com.mongodb.Mongo$2.execute(Mongo.java:828) ~[mongodb-driver-3.4.3.jar:na]
        at com.mongodb.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:338) ~[mongodb-driver-3.4.3.jar:na]
        at com.mongodb.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:322) ~[mongodb-driver-3.4.3.jar:na]
        at org.apache.camel.component.mongodb.MongoDbProducer.lambda$createDoInsert$7(MongoDbProducer.java:410) ~[camel-mongodb-2.20.0.jar:2.20.0]
        at org.apache.camel.component.mongodb.MongoDbProducer.lambda$wrap$0(MongoDbProducer.java:231) ~[camel-mongodb-2.20.0.jar:2.20.0]
        at org.apache.camel.component.mongodb.MongoDbProducer.invokeOperation(MongoDbProducer.java:112) ~[camel-mongodb-2.20.0.jar:2.20.0]
        at org.apache.camel.component.mongodb.MongoDbProducer.process(MongoDbProducer.java:96) ~[camel-mongodb-2.20.0.jar:2.20.0]
        ... 28 common frames omitted

Then I need to write:

 .marshal(jackson) 
        .convertBodyTo(String.class) 

but looks like producerTemplate doesn't have appropriate API.

Ok, I understand the cause of exception. @Neil Lunn said that it is because names started with $ - reserved.

2.

I created a topic Could not save date field as ISO date in mongo db via Camel (Caused by: java.lang.IllegalArgumentException: Invalid BSON field name $date)

According @Neil Lunn advices I tried to do

a) return Date in model:

public DateTime getCreatedDate() {
        return new DateTime(Date.from(createdDate.toInstant(ZoneOffset.UTC))).toDate();        
    }

It lead to saving date as NumberLong

b) tried to change $date with myDate in serializer but it lead to saving date as usual string

с)

Also I tried to change date format in serializer according @Neil Lunn advic in previous question:

private void serializeContents(Date value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
    jgen.writeFieldName("mydate");
    SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss");
    String formattedDate = formatter.format(value);
    jgen.writeString("ISODate("  +formattedDate + ")");
}

But it still doesn't work and I see:

"createdDate" : {
                "mydate" : "ISODate(2017-11-08T13:15:06)"
        },

in mongo-shell

Please, help to struggle this issue?

P.S.

When I use spring-data - date stores right now without additional movements