如何使用salesforce apex将较大的文件（大于12MB）上载到aws s3存储桶

Question

我需要一些帮助，以便从 salesforce 顶点服务器端将大文件上传到 s3 存储桶中。

我需要能够使用Http PUT操作拆分一个blob并将其上传到aws s3 bucket。我能够在一次上传中完成高达12MB的文件，因为这是Apex中PUT请求正文大小的限制。所以我需要能够使用多部分操作上传。我注意到s3允许部分上传并返回uploadId。想知道之前是否有人在salesforceapex代码中做过这项工作。我们将不胜感激。

提前感谢帕尔巴蒂·博斯。

这是代码。

public with sharing class AWSS3Service {

    private static Http http;

    @auraEnabled
    public static  void uploadToAWSS3( String fileToUpload , String filenm , String doctype){


        fileToUpload = EncodingUtil.urlDecode(fileToUpload, 'UTF-8');

        filenm = EncodingUtil.urlEncode(filenm , 'UTF-8'); // encode the filename in case there are special characters in the name 
        String filename = 'Storage' + '/' + filenm ;
        String attachmentBody = fileToUpload;

        String formattedDateString = DateTime.now().formatGMT('EEE, dd MMM yyyy HH:mm:ss z');




        // s3 bucket!
        String key = '**********' ;
        String secret = '********' ;
        String bucketname = 'testbucket' ;
        String region = 's3-us-west-2' ;



        String host = region + '.' + 'amazonaws.com' ; //aws server base url




    try{
        HttpRequest req = new HttpRequest();
        http = new Http() ;
        req.setMethod('PUT');
        req.setEndpoint('https://' + bucketname + '.' + host +  '/' +  filename );
        req.setHeader('Host', bucketname + '.' + host);

        req.setHeader('Content-Encoding', 'UTF-8');
        req.setHeader('Content-Type' , doctype);
        req.setHeader('Connection', 'keep-alive');
        req.setHeader('Date', formattedDateString);
        req.setHeader('ACL', 'public-read-write');


        String stringToSign = 'PUT\n\n' +
                doctype + '\n' +
                formattedDateString + '\n' +
                '/' + bucketname +  '/' + filename;


        Blob mac = Crypto.generateMac('HMACSHA1', blob.valueof(stringToSign),blob.valueof(secret));
        String signed = EncodingUtil.base64Encode(mac);
        String authHeader = 'AWS' + ' ' + key + ':' + signed;
        req.setHeader('Authorization',authHeader);

        req.setBodyAsBlob(EncodingUtil.base64Decode(fileToUpload)) ;



        HttpResponse response = http.send(req);

        Log.debug('response from aws s3 is ' + response.getStatusCode() + ' and ' + response.getBody());


    }catch(Exception e){
            Log.debug('error in connecting to s3 ' + e.getMessage());
            throw e ;
        }
    }

Answer 1

AWS SDK forJava公开了一个名为TransferManager的高级API，它简化了多部分上传（请参阅使用多部分上传API上传对象）。您可以从文件或流上传数据。您还可以设置高级选项，例如要用于多部分上传的部件大小，或者上传部件时要使用的并发线程数。您还可以设置可选的对象属性、储存类别或ACL。您可以使用PutObjectRequest和TransferManagerConfiguration类来设置这些高级选项。

这是https://docs.aws.amazon.com/AmazonS3/latest/dev/HLuploadFileJava.html.的示例代码

您可以调整您的Salesforce Apex代码:

import com.amazonaws.SdkClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.TransferManagerBuilder;
import com.amazonaws.services.s3.transfer.Upload;

import java.io.File;

public class HighLevelMultipartUpload {

    public static void main(String[] args) throws Exception {
        Regions clientRegion = Regions.DEFAULT_REGION;
        String bucketName = "*** Bucket name ***";
        String keyName = "*** Object key ***";
        String filePath = "*** Path for file to upload ***";

        try {
            AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                    .withRegion(clientRegion)
                    .withCredentials(new ProfileCredentialsProvider())
                    .build();
            TransferManager tm = TransferManagerBuilder.standard()
                    .withS3Client(s3Client)
                    .build();

            // TransferManager processes all transfers asynchronously,
            // so this call returns immediately.
            Upload upload = tm.upload(bucketName, keyName, new File(filePath));
            System.out.println("Object upload started");

            // Optionally, wait for the upload to finish before continuing.
            upload.waitForCompletion();
            System.out.println("Object upload complete");
        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process 
            // it, so it returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }
    }
}

Answer 2

前几天我也在看这个问题，不幸的是，由于APEX堆大小限制为12MB，您从Salesforce外部进行这种转移会更好。https://developer . sales force . com/docs/atlas . en-us . apex code . meta/apex code/apex _ gov _ limits . htm

虽然可以使用多部分写入文件，但似乎无法将它们从html" target="_blank">数据库中取出，以便将它们拆分为可以发送的块。在堆栈交换中提出了类似的问题 - https://salesforce.stackexchange.com/questions/264015/how-to-retrieve-file-content-from-content-document-in-chunks-using-soql