RestTemplate POSTing entity with associations to S

Question

The problem

I have three entities (taken from the Spring Data REST Exporter Example): Person, Address and Profile. A Person can have addresses and profiles.

@Entity
public class Person {

    @Id
    @GeneratedValue
    private Long id;
    private String name;
    @Version
    private Long version;
    @OneToMany
    private List<Address> addresses;
    @OneToMany
    private Map<String, Profile> profiles;

    // getters and setters
}

In the client side I use Spring's RestTemplate. I added the Jackson2HalModule to the ObjectMapper used by the MappingJackson2HttpMessageConverter used by my RestTemplate.

Since Address and Profile do not have references to other entities I can POST them to my Spring Data REST server, and they are successfully saved:

final ResponseEntity<Resource<Address>> response = restTemplate.postForEntity("http://localhost:8080/addresses",
                addressInstance, AddressResource.class);

where AddressResource extends org.springframework.hateoas.Resource<Address>.

But when I try to POST a Person instance

final ResponseEntity<Resource<Person>> response = restTemplate.postForEntity("http://localhost:8080/people",
                personInstance, PersonResource.class);

I get a org.springframework.web.client.HttpClientErrorException: 400 Bad Request and I think the cause is the associated Addresses and Profiles are serialized as normal POJOs instead as their resource URIs.

Here is the actual body of the POST request:

{
   "id":null,
   "name":"Jongjin Han",
   "version":null,
   "addresses":[
      {
         "id":1,
         "lines":[
            "1111",
            "coder's street"
         ],
         "city":"San Diego",
         "province":"California",
         "postalCode":"60707"
      },
      {
         "id":2,
         "lines":[
            "1111",
            "coder's street"
         ],
         "city":"San Diego",
         "province":"California",
         "postalCode":"60707"
      }
   ],
   "profiles":{
      "key1":{
         "type":"a type of profile",
         "url":"http://www.profileurl.com"
      },
      "key2":{
         "type":"a type of profile",
         "url":"http://www.profileurl.com"
      }
   }
}

I think it should be --> EDIT: It should be

{
   "id":null,
   "name":"Jongjin Han",
   "version":null,
   "addresses":[
      "http://localhost:8080/addresses/1",
      "http://localhost:8080/addresses/2"
   ],
   "profiles":{
      "key1":"http://localhost:8080/profiles/1",
      "key2":"http://localhost:8080/profiles/2"
   }
}

in fact the response body from the server is

{
  "cause" : {
    "cause" : {
      "cause" : {
        "cause" : null,
        "message" : "Cannot resolve URI id. Is it local or remote? Only local URIs are resolvable."
      },
      "message" : "Failed to convert from type java.net.URI to type org.springframework.data.rest.example.model.Address for value 'id'; nested exception is java.lang.IllegalArgumentException: Cannot resolve URI id. Is it local or remote? Only local URIs are resolvable."
    },
    "message" : "Failed to convert from type java.net.URI to type org.springframework.data.rest.example.model.Address for value 'id'; nested exception is java.lang.IllegalArgumentException: Cannot resolve URI id. Is it local or remote? Only local URIs are resolvable. (through reference chain: org.springframework.data.rest.example.model.Person[\"addresses\"]->java.util.ArrayList[1])"
  },
  "message" : "Could not read document: Failed to convert from type java.net.URI to type org.springframework.data.rest.example.model.Address for value 'id'; nested exception is java.lang.IllegalArgumentException: Cannot resolve URI id. Is it local or remote? Only local URIs are resolvable. (through reference chain: org.springframework.data.rest.example.model.Person[\"addresses\"]->java.util.ArrayList[1]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: Failed to convert from type java.net.URI to type org.springframework.data.rest.example.model.Address for value 'id'; nested exception is java.lang.IllegalArgumentException: Cannot resolve URI id. Is it local or remote? Only local URIs are resolvable. (through reference chain: org.springframework.data.rest.example.model.Person[\"addresses\"]->java.util.ArrayList[1])"
}

The possible solution I'd like to implement

Since I can access the REST repositories from the client side I am looking for a way to customize the Jackson Json Serializer in order to:

• check if the object I am serializing is a REST exported entity (easy with reflection, if only I could know where to put the code) and
• If I am serializing an entity, serialize the non-association fields as usual (e.g. person's name) and the association fields as their Resource URI (e.g. person's addresses) (with reflection it should be easy to convert from an entity to its resource URI, but I do not know where to put the code once again)

I tried with Jackson's JsonSerializer and PropertyFilters for Address and Profile, but I want a serializer which serialize them as resource URI only when they are in an association.

Any hint or aternative solution will be helpful.

Answer 1

Something is not configured correctly.

You should not have to POST HAL formatted data to get this to work, a plain old POJO serialized to JSON should work fine with the default configuration.

I would suggest using a proxy to intercept the request and confirm the structure.

Answer 2

I faced the same problem and tried to solve it with several techniques. Implemented working solution - it's a dirty workaround so don't blame me for the quality of the code, probably I will clean it up later :) I wanted to test Spring Data REST API and realized that MappingJackson2HttpMessageConverter ignores @Entity relations. Setting serializer modifier didn't work correctly: null-value serializer didn't work and relations serialized with deep property serialization.

The idea of workaround is to provide CustomSerializerModifier which returns CustomSerializer for project @Entities (inherited from BaseEntity in this example). CustomSerializer performs following actions:

1. write null-values (because omits them)
2. provide array of related @Entities as List in Spring Data REST style (//)
3. execute serialize(...) of default MappingJackson2HttpMessageConverter but providing NameTransformer which renames relations key (add ' _@ ') and then applies filter which excludes all fields starting with ' _@ '

I don't like this monster, but it works, and sadly I didn't find any solution :/

Working solution:

BasicRestTest

import com.fasterxml.jackson.core.Version;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.ser.FilterProvider;
import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider;
import com.meddis.util.serializer.CustomIgnorePropertyFilter;
import com.meddis.util.serializer.CustomSerializerModifier;
import org.junit.Before;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.hateoas.MediaTypes;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.mock.http.MockHttpOutputMessage;
import org.springframework.security.authentication.encoding.Md5PasswordEncoder;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.web.context.WebApplicationContext;

import java.io.IOException;
import java.nio.charset.Charset;

import static org.junit.Assert.assertNotNull;
import static org.springframework.test.web.servlet.setup.MockMvcBuilders.webAppContextSetup;

@RunWith(SpringRunner.class)
@ActiveProfiles({"test"})
@TestPropertySource(properties = {
        "timezone = UTC"
})
@SpringBootTest(classes = Application.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public abstract class BasicRestTest {

    protected String host = "localhost";

    @Value("${local.server.port}")
    protected int port;

    @Value("${spring.data.rest.basePath}")
    protected String springDataRestBasePath;

    protected MediaType contentType = new MediaType("application", "hal+json", Charset.forName("utf8"));

    protected MockMvc mockMvc;

    private static HttpMessageConverter mappingJackson2HttpMessageConverter;

    protected ObjectMapper objectMapper;

    @Autowired
    private WebApplicationContext webApplicationContext;

    @Autowired
    void setConverters(HttpMessageConverter<?>[] converters) {

        this.objectMapper = new ObjectMapper();

        if (this.mappingJackson2HttpMessageConverter == null) {
            this.mappingJackson2HttpMessageConverter = new MappingJackson2HttpMessageConverter();
            SimpleModule simpleModule = new SimpleModule("CUSTOM", Version.unknownVersion());
            simpleModule.setSerializerModifier(new CustomSerializerModifier(springDataRestBasePath));
            ((MappingJackson2HttpMessageConverter) this.mappingJackson2HttpMessageConverter).getObjectMapper()
                    .registerModule(simpleModule);

            FilterProvider fp = new SimpleFilterProvider().addFilter("CUSTOM", new CustomIgnorePropertyFilter());
            ((MappingJackson2HttpMessageConverter) this.mappingJackson2HttpMessageConverter).getObjectMapper()
                    .setFilterProvider(fp);

            ((MappingJackson2HttpMessageConverter) this.mappingJackson2HttpMessageConverter).setPrettyPrint(true);
        }

        assertNotNull("the JSON message converter must not be null", this.mappingJackson2HttpMessageConverter);
    }

    @Before
    public void setup() throws Exception {
        this.mockMvc = webAppContextSetup(webApplicationContext).build();
    }

    protected String json(final Object o) throws IOException {
        MockHttpOutputMessage mockHttpOutputMessage = new MockHttpOutputMessage();
        this.mappingJackson2HttpMessageConverter.write(o, MediaTypes.HAL_JSON, mockHttpOutputMessage);
        return mockHttpOutputMessage.getBodyAsString();
    }
}

CustomSerializerModifier

import com.fasterxml.jackson.databind.BeanDescription;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializationConfig;
import com.fasterxml.jackson.databind.ser.BeanSerializerModifier;
import com.meddis.model.BaseEntity;

public class CustomSerializerModifier extends BeanSerializerModifier {

    private final String springDataRestBasePath;

    public CustomSerializerModifier(final String springDataRestBasePath) {
        this.springDataRestBasePath = springDataRestBasePath;
    }

    @Override
    public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription beanDesc, JsonSerializer<?> serializer) {
        if (BaseEntity.class.isAssignableFrom(beanDesc.getBeanClass())) {
            return new CustomSerializer((JsonSerializer<Object>) serializer, springDataRestBasePath);
        }
        return serializer;
    }
}

CustomSerializer

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.util.NameTransformer;
import com.google.common.base.Preconditions;
import com.meddis.model.BaseEntity;

import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.util.*;

public class CustomSerializer extends JsonSerializer<Object> {

    private final JsonSerializer<Object> defaultSerializer;

    private final String springDataRestBasePath;

    public CustomSerializer(JsonSerializer<Object> defaultSerializer, final String springDataRestBasePath) {
        this.defaultSerializer = Preconditions.checkNotNull(defaultSerializer);
        this.springDataRestBasePath = springDataRestBasePath;
    }

    @SuppressWarnings("unchecked")
    @Override
    public void serialize(Object baseEntity, JsonGenerator jsonGenerator, SerializerProvider serializerProvider)
            throws IOException, JsonProcessingException {

        jsonGenerator.writeStartObject();

        Set<String> nestedEntityKeys = new HashSet<>();

        Arrays.asList(baseEntity.getClass().getMethods()).stream()
                .filter(field -> field.getName().startsWith("get"))
                .filter(field -> !Arrays.asList("getClass", "getVersion").contains(field.getName()))
                .forEach(field -> {
                    try {
                        Object value = field.invoke(baseEntity, new Object[]{});
                        String fieldName = field.getName().replaceAll("^get", "");
                        fieldName = fieldName.substring(0, 1).toLowerCase() + fieldName.substring(1);
                        if (value == null) {
                            jsonGenerator.writeObjectField(fieldName, null);
                        } else if (Iterable.class.isAssignableFrom(value.getClass())) {
                            Iterator it = ((Iterable) value).iterator();
                            // System.out.println(field.getName() + field.invoke(baseEntity, new Object[]{}));
                            List<String> nestedUris = new ArrayList<>();
                            it.forEachRemaining(nestedValue -> {
                                if (BaseEntity.class.isAssignableFrom(nestedValue.getClass())) {
                                    try {
                                        String nestedEntityStringDataName = nestedValue.getClass().getSimpleName() + "s";
                                        nestedEntityStringDataName = nestedEntityStringDataName.substring(0, 1).toLowerCase() + nestedEntityStringDataName.substring(1);
                                        Long nestedId = (long) nestedValue.getClass().getMethod("getId").invoke(nestedValue, new Object[]{});
                                        String nestedEntitySpringDataPath = springDataRestBasePath + "/" + nestedEntityStringDataName + "/" + Long.toString(nestedId);
                                        nestedUris.add(nestedEntitySpringDataPath);
                                    } catch (IllegalAccessException | InvocationTargetException | NoSuchMethodException ignored) {
                                    }
                                }
                            });
                            nestedEntityKeys.add(fieldName);
                            jsonGenerator.writeObjectField(fieldName, nestedUris);
                        }
                    } catch (Throwable ignored) {
                    }
                });

        // Apply default serializer
        ((JsonSerializer<Object>) defaultSerializer.unwrappingSerializer(new NameTransformer() {
            @Override
            public String transform(String s) {
                if (nestedEntityKeys.contains(s)) {
                    return "_@" + s;
                }
                return s;
            }

            @Override
            public String reverse(String s) {
                if (nestedEntityKeys.contains(s.substring(2))) {
                    return s.substring(2);
                }
                return s;
            }
        }).withFilterId("CUSTOM")).serialize(baseEntity, jsonGenerator, serializerProvider);

        jsonGenerator.writeEndObject();
    }
}

CustomIgnorePropertyFilter

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.jsonFormatVisitors.JsonObjectFormatVisitor;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.fasterxml.jackson.databind.ser.PropertyWriter;
import com.fasterxml.jackson.databind.ser.impl.SimpleBeanPropertyFilter;

public class CustomIgnorePropertyFilter extends SimpleBeanPropertyFilter {

    @Override
    public void serializeAsField(Object o, JsonGenerator jsonGenerator, SerializerProvider serializerProvider, PropertyWriter propertyWriter) throws Exception {
        if (propertyWriter.getName().startsWith("_@")) {
            return;
        }
        super.serializeAsField(o, jsonGenerator, serializerProvider, propertyWriter);
    }

    @Override
    public void serializeAsElement(Object o, JsonGenerator jsonGenerator, SerializerProvider serializerProvider, PropertyWriter propertyWriter) throws Exception {
        if (propertyWriter.getName().startsWith("_@")) {
            return;
        }
        super.serializeAsElement(o, jsonGenerator, serializerProvider, propertyWriter);
    }

    @Override
    public void depositSchemaProperty(PropertyWriter propertyWriter, ObjectNode objectNode, SerializerProvider serializerProvider) throws JsonMappingException {
        if (propertyWriter.getName().startsWith("_@")) {
            return;
        }
        super.depositSchemaProperty(propertyWriter, objectNode, serializerProvider);
    }

    @Override
    public void depositSchemaProperty(PropertyWriter propertyWriter, JsonObjectFormatVisitor jsonObjectFormatVisitor, SerializerProvider serializerProvider) throws JsonMappingException {
        if (propertyWriter.getName().startsWith("_@")) {
            return;
        }
        super.depositSchemaProperty(propertyWriter, jsonObjectFormatVisitor, serializerProvider);
    }
}

VideoStreamRestTest

import com.meddis.AdminApiTest;
import com.meddis.model.VideoStream;
import com.meddis.repository.SpecialistRepository;
import com.meddis.repository.VideoStreamTagRepository;
import org.junit.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.web.servlet.MvcResult;

import java.util.stream.Collectors;
import java.util.stream.StreamSupport;

import static org.hamcrest.Matchers.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

/**
 * <a href="https://spring.io/guides/tutorials/bookmarks/">example</a>
 */
public class VideoStreamRestTest extends AdminApiTest {

    @Autowired
    private SpecialistRepository specialistRepository;

    @Autowired
    private VideoStreamTagRepository videoStreamTagRepository;

    @Test
    public void springDataRestVideoStreams() throws Exception {
        String requestBody;
        String newEntityTitle = md5("VIDEO_STREAM_");
        MvcResult create = mockMvc.perform(post(springDataRestBasePath + "/videoStreams").headers(authenticationHeader)
                .content(requestBody = json(new VideoStream()
                        .setTitle(newEntityTitle)
                        .setType(VideoStream.Type.BROADCAST)
                        .setPrice(10.0)
                        .setDurationInMinutes(70)
                        .setDescription("broadcast description")
                        .setPreviewUrl("http://example.com")
                        .setSpecialists(StreamSupport.stream(specialistRepository.findAll().spliterator(), false).collect(Collectors.toList()))
                        .setTags(StreamSupport.stream(videoStreamTagRepository.findAll().spliterator(), false).collect(Collectors.toList())))))
                .andExpect(status().isCreated())
                .andReturn();
        String createdLocation = create.getResponse().getHeader("Location");
        logger.info("Created new entity: {}", createdLocation);
        logger.info("Sent: {}", requestBody);

        MvcResult list = mockMvc.perform(get(springDataRestBasePath + "/videoStreams").headers(authenticationHeader))
                .andExpect(status().isOk())
                .andExpect(content().contentType(contentType))
                .andExpect(jsonPath("$._embedded.videoStreams", hasSize(greaterThanOrEqualTo(1))))
                .andExpect(jsonPath("$._embedded.videoStreams[*].title", hasItem(newEntityTitle)))
                .andExpect(jsonPath("$._embedded.videoStreams[*]._links.self.href", hasItem(createdLocation)))
                .andReturn();
        logger.info("Got list containing new entity:\n{}", list.getResponse().getContentAsString());

        MvcResult createdEntity = mockMvc.perform(get(createdLocation).headers(authenticationHeader))
                .andExpect(status().isOk())
                .andExpect(jsonPath("$._links.self.href", equalTo(createdLocation)))
                .andExpect(jsonPath("$.title", equalTo(newEntityTitle)))
                .andReturn();
        logger.info("Got new entity:\n{}", createdEntity.getResponse().getContentAsString());
    }

}

AdminApiTest

import com.fasterxml.jackson.databind.JsonNode;
import org.junit.Before;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.test.web.servlet.MvcResult;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import static org.junit.Assert.assertEquals;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

public abstract class AdminApiTest extends BasicRestTest {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    protected HttpHeaders authenticationHeader;

    @Before
    @Override
    public void setup() throws Exception {
        super.setup();
        this.authenticationHeader = createHeaderWithAuthentication();
    }

    protected HttpHeaders createHeaderWithAuthentication() throws IOException {
        String user = "[e-mail]";
        String password = "pasha";
        ResponseEntity<String> response = new TestRestTemplate()
                .postForEntity(
                        "http://" + host + ":" + port
                                + "login?"
                                + "&username=" + user
                                + "&password=" + password,
                        null,
                        String.class
                );
        assertEquals(HttpStatus.FOUND, response.getStatusCode());
        List<String> authenticationCookie = response.getHeaders().get("Set-Cookie");
        assertEquals(1, authenticationCookie.size());
        HttpHeaders headers = new HttpHeaders();
        headers.set("Cookie", authenticationCookie.get(0));
        return headers;
    }

}