试图把InputStream的亚马逊S3时,进程死亡(Process dies when tryin

2019-06-27 22:48发布

这是我做的写的InputStream

public OutputStream getOutputStream(@Nonnull final String uniqueId) throws PersistenceException {
        final PipedOutputStream outputStream = new PipedOutputStream();
        final PipedInputStream inputStream;
        try {
            inputStream = new PipedInputStream(outputStream);
            new Thread(
                    new Runnable() {
                        @Override
                        public void run() {
                            PutObjectRequest putObjectRequest = new PutObjectRequest("haritdev.sunrun", "sample.file.key", inputStream, new ObjectMetadata());
                            PutObjectResult result = amazonS3Client.putObject(putObjectRequest);
                            LOGGER.info("result - " + result.toString());
                            try {
                                inputStream.close();
                            } catch (IOException e) {

                            }
                        }
                    }
            ).start();
        } catch (AmazonS3Exception e) {
            throw new PersistenceException("could not generate output stream for " + uniqueId, e);
        } catch (IOException e) {
            throw new PersistenceException("could not generate input stream for S3 for " + uniqueId, e);
        }
         try {
            return new GZIPOutputStream(outputStream);
        } catch (IOException e) {
            LOGGER.error(e.getMessage(), e);
            throw new PersistenceException("Failed to get output stream for " + uniqueId + ": " + e.getMessage(), e);
        }
    }

而在下面的方法,我看到我的工艺模具

 protected <X extends AmazonWebServiceRequest> Request<X> createRequest(String bucketName, String key, X originalRequest, HttpMethodName httpMethod) { Request<X> request = new DefaultRequest<X>(originalRequest, Constants.S3_SERVICE_NAME); request.setHttpMethod(httpMethod); if (bucketNameUtils.isDNSBucketName(bucketName)) { request.setEndpoint(convertToVirtualHostEndpoint(bucketName)); request.setResourcePath(ServiceUtils.urlEncode(key)); } else { request.setEndpoint(endpoint); if (bucketName != null) { /* * We don't URL encode the bucket name, since it shouldn't * contain any characters that need to be encoded based on * Amazon S3's naming restrictions. */ request.setResourcePath(bucketName + "/" + (key != null ? ServiceUtils.urlEncode(key) : "")); } } return request; } 

该过程对request.setResourcePath(ServiceUtils.urlEncode(键))失败; 我甚至不能调试正因为如此,即使关键是有效的名称,而不是

是否有人可以帮助?

这是怎么request临终前看

request = {com.amazonaws.DefaultRequest@1931}"PUT https://my.bucket.s3.amazonaws.com / "
resourcePath = null
parameters = {java.util.HashMap@1959} size = 0
headers = {java.util.HashMap@1963} size = 0
endpoint = {java.net.URI@1965}"https://my.bucket.s3.amazonaws.com"
serviceName = {java.lang.String@1910}"Amazon S3"
originalRequest = {com.amazonaws.services.s3.model.PutObjectRequest@1285}
httpMethod = {com.amazonaws.http.HttpMethodName@1286}"PUT"
content = null

Answer 1:

我尝试了同样的方法,但失败对我来说也是如此。

我结束了我的所有数据写入到输出流,然后再从输出数据流的输入流复制数据后,启动上载至S3:

...
// Data written to outputStream here
...
byte[] byteArray = outputStream.toByteArray();
amazonS3Client.uploadPart(new UploadPartRequest()
  .withBucketName(bucket)
  .withKey(key)
  .withInputStream(new ByteArrayInputStream(byteArray))
  .withPartSize(byteArray.length)
  .withUploadId(uploadId)
  .withPartNumber(partNumber));

那种失败写入流,如果整个数据块已被写入和上传到S3,甚至可以开始前要在内存复制的目的,但它是我能得到它的工作的唯一途径。



Answer 2:

这里是我的尝试和努力 -

  try (PipedOutputStream pipedOutputStream = new PipedOutputStream();
     PipedInputStream pipedInputStream = new PipedInputStream()) {
            new Thread(new Runnable() {

            public void run() {
                try {
                      // write some data to pipedOutputStream
                } catch (IOException e) {
                   // handle exception
                }
            }
            }).start();
       PutObjectRequest putObjectRequest = new PutObjectRequest(BUCKET, FILE_NAME, pipedInputStream, new ObjectMetadata());
       s3Client.putObject(putObjectRequest);
}

此代码曾与S3投掷警告,内容长度未设置和S3将被缓冲,并可能导致OutOfMemoryException异常。 我不相信关于设置内容长度在ObjectMetadata只是为了摆脱这一消息,并希望AWS SDK不会流全流进内存只是为了找到内容长度的任何便宜的方法。



文章来源: Process dies when trying to put InputStream to Amazon S3