Spring boot native image is not able to convert 'SORT' to json, throws org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON

[pulsar-gupta]

I am generating spring boot 3 native image using -Pnative switch.
The DAO class fetches the data from DB with SORT object passed to DAO (below is sample)

Sort sort = sortType.equalsIgnoreCase(Sort.Direction.ASC.name()) ? Sort.by(sortField).ascending()
					: Sort.by(sortField).descending();
			Pageable pageWithElements = PageRequest.of(pageNum, projectsCount,sort);
			Page<ProjectDetailsListDto> result= projectDetailsDao.getProjectList(pageWithElements);

However, the call is failed when the call is returned from controller and the exception is

"org.springframework.web.servlet.mvc.support.DefaultHandlerExceptionResolver.logException(207) : Resolved [org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: Couldn't serialize object"

I have tried to provide hints to serialize the SORT object but this is also not working.

[wilkinsona]

Thanks for the report. Unfortunately, without knowing what type you are returning in the HTTP response and exactly what hints you provided, we can only guess about the cause of the problem. If you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.

[pulsar-gupta]

master-service.zip

Thank you for the quick response. Attached the code for your reference.
HTTP request : http://localhost:8080/master/project/list?pageNum=0&projectsCount=10&sortType=DESC&sortField=projectStatus

[wilkinsona]

Thanks for the sample. Unfortunately, I haven't been able to reproduce the problem. I am trying to do so on the JVM initially. Once that works, I'll try in a native image.

The project requires Postgres which I'd prefer not to use as there may be external configuration in the database that you haven't shared. I switched to H2 which allowed the app to start.

Accessing http://localhost:8080/master/project/list?pageNum=0&projectsCount=10&sortType=DESC&sortField=projectStatus results in a 404 response. Looking at the controllers, I think the correct URL is http://localhost:8080/list?pageNum=0&projectsCount=10&sortType=DESC&sortField=projectStatus. This results in a different 404 response:

HTTP/1.1 404 
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 01 Jun 2023 13:41:35 GMT

{"message":"Error while fetching project list","timestamp":"2023-06-01T14:41:35.542967","errorCode":"NOT_FOUND"}

With the following logged to the console:

2023-06-01 14:41:35,542 ERROR 32681 --- [http-nio-8080-exec-5] ERROR [master-service, 092b8de4642b945a09aed6a460e50a75, b1f64cb77d7b2d92]  org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(138) : Table "PLANNING_LEVEL" not found; SQL statement:
select m.id, m.project_name as projectName,m.internal_project_id as internalProjectId,
l.planning_level as planningLevel,s.project_status as projectStatus,
p.name as portfolioUnitName FROM(( 
masterdata m	INNER JOIN  planning_level l
on m.planning_level=l.id )
INNER JOIN project_status s
on m.project_status=s.id
INNER JOIN portfolio_unit p on
m.portfolio_unit_id=p.portfolio_unit_id) order by projectStatus desc fetch first ? rows only [42102-214]
2023-06-01 14:41:35,542 ERROR 32681 --- [http-nio-8080-exec-5] ERROR [master-service, 092b8de4642b945a09aed6a460e50a75, b1f64cb77d7b2d92]  com.serviceImpl.ProjectServiceImpl.getProjectsList(43) : Error while fetching project listcould not prepare statement; SQL [select m.id, m.project_name as projectName,m.internal_project_id as internalProjectId,
l.planning_level as planningLevel,s.project_status as projectStatus,
p.name as portfolioUnitName FROM(( 
masterdata m	INNER JOIN  planning_level l
on m.planning_level=l.id )
INNER JOIN project_status s
on m.project_status=s.id
INNER JOIN portfolio_unit p on
m.portfolio_unit_id=p.portfolio_unit_id) order by projectStatus desc fetch first ? rows only]

It looks like the project's entities are not sufficient for Hibernate to create the expected database schema.

Can you please update the sample to use an in-memory database such as H2 and ensure that it reproduces the problem you have described without relying on any additional external configuration.

[pulsar-gupta]

I have uploaded the app with h2 enabled (demo.zip).
api: http://localhost:8080/list?pageNum=0&projectsCount=1&sortType=DESC&sortField=projectStatus
swagger: http://localhost:8080/swagger-ui/index.html#

demo.zip

Let me know If i can be a help

[wilkinsona]

Thanks for the updates. Unfortunately, I still can't reproduce the problem. With Spring Boot 3.1.0, the native image fails to start due to some known problems with Hibernate 6.2.2. When I downgrade to Spring Boot 3.0.7, the image starts and the HttpMessageNotWritableException does not occur:

$ curl 'http://localhost:8080/list?pageNum=0&projectsCount=1&sortType=DESC&sortField=projectStatus' 
{"totalPages":0,"totalElements":0,"pageable":{"pageSize":1,"pageNumber":0,"paged":true,"unpaged":false,"offset":0,"sort":{"unsorted":false,"sorted":true,"empty":false}},"numberOfElements":0,"size":1,"content":[],"sort":{"unsorted":false,"sorted":true,"empty":false},"number":0,"first":true,"last":true,"empty":true}

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-projects-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

[pulsar-gupta]

I was facing this issue because of feign dependency in application. I was not able to find the root cause of it, but as soon as I removed all the feign dependencies the sort started to work in native image.

Thank you for the support.

[wilkinsona]

The sample you provided includes a dependency on Feign but it does not appear to reproduce the problem. Unfortunately, without something that reproduces the problem we won't be able to help.

[mykhailokulakov]

Hi @pulsar-gupta @wilkinsona
I also faced this issue using PageImpl + openfeign, and it was reproducible on the native build only (regular build works fine).

spring-boot-starter 3.2.0
spring-cloud-dependencies 2023.0.0

It's enough to return a dummy PageImpl in your controller to raise this exception:

    @GetMapping("/sort_issue")
    @PageableAsQueryParam
    Page<Model> testException(@Parameter(hidden = true) Pageable pageable) {
        return new PageImpl<>(List.of(), pageable, 0);
    }

The request should contain the sort field: search?page=0&size=20&sort=createdAt%2Casc
Exception:

<#4f04abcf> o.s.http.converter.HttpMessageNotWritableException: Could not write JSON: Couldn't serialize object createdAt: ASC
	at o.s.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:492)
	... 12 frames excluded
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	... 72 frames excluded
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	... 2 frames excluded
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	... 2 frames excluded
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	... 2 frames excluded
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	... 2 frames excluded
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:340)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.runWith(Thread.java:1596)
	at java.lang.Thread.run(Thread.java:1583)
	at com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:837)
	at com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:211)
Caused by: <#cc5dd32d> com.fasterxml.jackson.databind.JsonMappingException: Couldn't serialize object createdAt: ASC (through reference chain: org.springframework.data.domain.PageImpl["pageable"]->org.springframework.data.domain.PageRequest["sort"])
	at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:323)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:780)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
	at c.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:479)
	at c.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:318)
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1572)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1061)
	... 1 frames excluded
	... 130 common frames omitted
Caused by: <#cc2d0664> feign.codec.EncodeException: Couldn't serialize object createdAt: ASC
	at o.s.cloud.openfeign.support.SortJsonComponent$SortSerializer.lambda$serialize$0(SortJsonComponent.java:54)
	at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:1085)
	... 2 frames excluded
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
	... 139 common frames omitted

It seems it's related to a misplaced serializator from openfeign
See spring-cloud/spring-cloud-openfeign#675

I was able to fix this by providing serialization config for the Sort class:

  import org.springframework.data.domain.Sort;

  @Bean
  public Jackson2ObjectMapperBuilderCustomizer jacksonObjectMapperCustomization() {
      return jacksonObjectMapperBuilder ->
              jacksonObjectMapperBuilder
                     ...
                      .serializerByType(Sort.class, ToStringSerializer.instance);
  }

Hope this tip will help someone!
Cheers

[pulsar-gupta]

@mykhailokulakov
Thanks ! I will try the solution.