Skip to content

After few failure retries and then sucessful, an empty response body returns #75

Open
@instcode

Description

@instcode

When I'm using Jetty92RetryHelper to support retry in my input plugin, if the retry fails several times and then the last one is successful, the request returns an empty body and causes the "no content..." error:

2017-04-12 20:01:31.715 +0700 [WARN] (main): Retrying 1/6 after 60 seconds. Message: Response not 2xx: 429 {"code": 429, "error": "requests rate exceed limit"}
2017-04-12 20:02:33.514 +0700 [WARN] (main): Retrying 2/6 after 120 seconds. Message: Response not 2xx: 429 UNKNOWN STATUS CODE 
2017-04-12 20:05:06.523 +0000 [INFO] (0001:transaction): {done:  1 / 1, running: 0}
2017-04-12 20:05:06.625 +0000 [WARN] (main): com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input
 at [Source: ; line: 1, column: 1]
org.embulk.exec.PartialExecutionException: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input
 at [Source: ; line: 1, column: 1]
	at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:373) ~[embulk-core-0.8.18.jar:na]

Please note that after 2 retries, the last retry was successful but the response body is empty. After adding some exception logging, I can see the real error is:

java.util.concurrent.ExecutionException: java.io.EOFException: HttpConnectionOverHTTP@4228f0d7(l:/192.168.0.118:37648 <-> r:xxx.xxxxxxxx.com/xx.xxx.xxx.xxx:443,closed=false)[HttpChannelOverHTTP@4d8e2c06(exchange=HttpExchange@6f1906b3 req=PENDING/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@6abee8e4(req=HEADERS,snd=SENDING,failure=null)[HttpGenerator{s=COMPLETING}],recv=HttpReceiverOverHTTP@6f95132c(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of 0}]]]
	at org.eclipse.jetty.client.util.InputStreamResponseListener.get(InputStreamResponseListener.java:228)
	at org.embulk.util.retryhelper.jetty92.StringJetty92ResponseEntityReader.getResponse(StringJetty92ResponseEntityReader.java:31)
	at org.embulk.util.retryhelper.jetty92.Jetty92RetryHelper$1.call(Jetty92RetryHelper.java:107)
	at org.embulk.spi.util.RetryExecutor.run(RetryExecutor.java:100)
	at org.embulk.spi.util.RetryExecutor.runInterruptible(RetryExecutor.java:77)
	at org.embulk.util.retryhelper.jetty92.Jetty92RetryHelper.requestWithRetry(Jetty92RetryHelper.java:95)
	at org.embulk.input.xxxxxxxxx.RetryableXXXXXXXWs.fetch(RetryableXXXXXXXWs.java:58)
	at org.embulk.base.restclient.RestClientInputPluginBaseUnsafe.run(RestClientInputPluginBaseUnsafe.java:110)
	at org.embulk.base.restclient.RestClientInputPluginBase.run(RestClientInputPluginBase.java:115)
	at org.embulk.base.restclient.RestClientInputPluginBaseUnsafe.resume(RestClientInputPluginBaseUnsafe.java:92)
	at org.embulk.base.restclient.RestClientInputPluginBase.resume(RestClientInputPluginBase.java:103)
	at org.embulk.base.restclient.RestClientInputPluginBaseUnsafe.transaction(RestClientInputPluginBaseUnsafe.java:85)
	at org.embulk.base.restclient.RestClientInputPluginBase.transaction(RestClientInputPluginBase.java:97)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.embulk.EmbulkTestRuntime$1$1.run(EmbulkTestRuntime.java:90)
	at org.embulk.EmbulkTestRuntime$1$1.run(EmbulkTestRuntime.java:86)
	at org.embulk.spi.Exec.doWith(Exec.java:25)
	at org.embulk.EmbulkTestRuntime$1.evaluate(EmbulkTestRuntime.java:86)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.io.EOFException: HttpConnectionOverHTTP@4228f0d7(l:/192.168.0.118:37648 <-> r:api.xxxxxxxx.com/xxx.xx.xx.xx:443,closed=false)[HttpChannelOverHTTP@4d8e2c06(exchange=HttpExchange@6f1906b3 req=PENDING/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@6abee8e4(req=HEADERS,snd=SENDING,failure=null)[HttpGenerator{s=COMPLETING}],recv=HttpReceiverOverHTTP@6f95132c(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of 0}]]]
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:277)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1309)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:182)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:129)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
	at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:89)
	at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:122)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)

Not sure if this is an issue with the Jetty92 or just specific to my plugin?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions