我们的应用程序在将数据上传到 S3 存储桶时遇到了问题。此上传在 Centos-6 实例上适用于同一程序,但在 RHEL7 实例上失败。
我们有带有附加角色水晶角色的EC2实例。反过来,此角色具有内联策略水晶策略:
{
"RoleName": "crystal-role",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:DescribeInstances"
],
"Resource": "*",
"Effect": "Allow",
"Sid": "AllowDescribeInstances"
},
{
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::crystal-dyn"
],
"Effect": "Allow",
"Sid": "AllowSeeLogBucket"
},
{
"Action": [
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::crystal-dyn/*"
],
"Effect": "Allow",
"Sid": "AllowPutLogs"
},
{
"Action": [
"kms:Encrypt",
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:us-east-1:566:key/a15912a107bb",
"arn:aws:kms:us-east-1:566201213358:key/158d81e9467a"
],
"Effect": "Allow",
"Sid": "AllowEncrypt"
},
{
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::389203956472:role/allow-cross-account-exec-api-qa2",
"Effect": "Allow",
"Sid": "AllowApiAccess"
}
]
},
"PolicyName": "crystal-policy"
}
此策略应该允许在实例上运行的Java应用程序将数据上传到S3存储桶。但是,这不会发生。因此,我们创建了一个简单的Java程序来测试上传。请注意,我使用了3个版本的创建Amazon S3客户端。我从命令行运行它:
java -cp ".:lib/*" org.examples.UploadObject
在lib中,我有以下jar:
aws-java-sdk-1.10.10.jar aws-java-sdk-s3-1.11.339.jar httpclient-4.5.5.jar jackson-annotations-2.9.5.jar jackson-databind-2.9.5.jar
aws-java-sdk-core-1.11.423.jar commons-logging-1.1.3.jar httpcore-4.4.9.jar jackson-core-2.9.5.jar joda-time-2.9.9.jar
Java代码:
String clientRegion = "us-east-1";
String bucketName = "crystal-dyn";
String stringObjKeyName = "stringToUploadTest";
try {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(DefaultAWSCredentialsProviderChain.getInstance()).withRegion(clientRegion).build();
System.out.println("s3Client=" + s3Client);
s3Client.putObject(bucketName, stringObjKeyName, "Uploaded String Object");
System.out.println("Uploading String is done");
}
catch(AmazonServiceException e) {
e.printStackTrace();
}
catch(SdkClientException e) {
e.printStackTrace();
}
try {
System.out.println("Uploading to S3 bucket=" + bucketName + " string=" + stringObjKeyName + " Building with No Creds");
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion(clientRegion).build();
System.out.println("s3Client=" + s3Client);
s3Client.putObject(bucketName, stringObjKeyName, "Uploaded String Object");
System.out.println("Uploading String is done");
}
catch(AmazonServiceException e) {
e.printStackTrace();
}
catch(SdkClientException e) {
e.printStackTrace();
}
try {
System.out.println("Uploading to S3 bucket=" + bucketName + " string=" + stringObjKeyName + " Building with No Creds and No region");
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build();
System.out.println("s3Client=" + s3Client);
s3Client.putObject(bucketName, stringObjKeyName, "Uploaded String Object");
System.out.println("Uploading String is done");
}
catch(AmazonServiceException e) {
e.printStackTrace();
}
catch(SdkClientException e) {
e.printStackTrace();
}
对于创建Amazon S3客户端的所有3个版本,此程序只有在从EC2Linux实例上的root用户运行时才能成功运行。对于所有其他用户,我们获得了带有异常堆栈的输出,请参阅下文。请注意,包括rood在内的任何用户都没有带有凭据的. aws目录。
s3Client=com.amazonaws.services.s3.AmazonS3Client@682b2fa
com.amazonaws.SdkClientException: Unable to load AWS credentials from
any provider in the chain: [EnvironmentVariableCredentialsProvider:
Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), com.amazonaws.auth.profile.ProfileCredentialsProvider@20d525: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3f56875e: Unable to load credentials from service endpoint]
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1186)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:776)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1755)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:3448)
at org.examples.UploadObject.main(UploadObject.java:20)
Uploading to S3 bucket=dynarch-ac-logs-malachite-dyn
string=stringToUploadTest Building with No Creds
s3Client=com.amazonaws.services.s3.AmazonS3Client@740773a3
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AF2735F42CCB60D0; S3 Extended Request ID: 7VyScO6XOs00oB/g0k8bqG3X3Ib01n4uT1xg8/2U72TCOKg8YKNIVgQrjjnF6XzUAfoB24wcYZY=), S3 Extended Request ID: 7VyScO6XOs00oB/g0k8bqG3X3Ib01n4uT1xg8/2U72TCOKg8YKNIVgQrjjnF6XzUAfoB24wcYZY=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1660)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1324)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1074)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1755)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:3448)
at org.examples.UploadObject.main(UploadObject.java:33)
Uploading to S3 bucket=dynarch-ac-logs-malachite-dyn
string=stringToUploadTest Building with No Creds and No region
com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.
at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:436)
at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:402)
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at org.examples.UploadObject.main(UploadObject.java:44)
问题似乎出在只允许访问根的 iptables 规则中。
iptables -L | grep root
DROP all -- anywhere instance-data.ec2.internal ! owner UID match root
删除此规则后,上传适用于所有用户
我只是在想,而不是删除规则。如果问题得到解决,您可以尝试以下操作吗?(如果我有你的示例项目,我想自己测试一下)。通常,我使用以下步骤来设置在没有root权限的情况下与S3交互的容器,并使用IAM角色。
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-install.html
sudo sh -c "echo 'net.ipv4.conf.all.route_localnet = 1' >> /etc/sysctl.conf"
sudo sysctl -p /etc/sysctl.conf
sudo iptables -t nat -A PREROUTING -p tcp -d 169.254.170.2 --dport 80 -j DNAT --to-destination 127.0.0.1:51679
sudo iptables -t nat -A OUTPUT -d 169.254.170.2 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 51679
sudo sh -c 'iptables-save > /etc/iptables/rules.v4'
//OR
sudo sh -c 'iptables-save > /etc/sysconfig/iptables'
提前致谢
问题内容: 假设一个项目包含几个类,每个类都有一个静态初始化程序块。这些块以什么顺序运行?我知道在一个类中,这样的块按照它们在代码中出现的顺序运行。我读过所有类都一样,但是我编写的一些示例代码对此表示不同。我使用以下代码: 并得到以下输出: START static - grandparent static - parent static - child instance - grandparen
我对使用AWS非常陌生,我正在将一些现有应用迁移到AWS,我看到的所有教程都有以下步骤: 启动EC2实例 一旦SSH会话丢失或终止,应用程序就会停止运行。 我如何才能保持应用程序始终运行和可用?
本文向大家介绍为什么Java中只有值传递?相关面试题,主要包含被问及为什么Java中只有值传递?时的应答技巧和注意事项,需要的朋友参考一下 首先回顾一下在程序设计语言中有关将参数传递给方法(或函数)的一些专业术语。按值调用(call by value)表示方法接收的是调用者提供的值,而按引用调用(call by reference)表示方法接收的是调用者提供的变量地址。一个方法可以修改传递引用所对
问题内容: 以下简单的Java代码: 使用14个线程运行。我知道在后台运行一些GC线程,但是其他线程又有什么用?为什么会有这么多线程?我在使用Java 1.6.0_26的Gentoo Linux上。使用Eclipse的编译器或javac进行编译没有任何区别(在Eclipse的调试模式下运行它会增加3个线程,但这可能是合理的)。 问题答案: 默认情况下,我的JVM(1.6.0_26)产生更多线程。大
在对Clojure应用进行基准测试并试图确定性能问题时,我注意到了这种特殊的行为:即使整个程序是用Java编写的,当从莱宁根启动时,它似乎会经历显著的减速。 假设我有这个Java程序: 当我简单地运行程序时,执行时间(每次)大约为1s。但是,当我从leiningen运行它时,就像这样: 我得到大约2秒的运行时间!Clojure/Leiningen是如何将一个完整的Java程序的速度降低这么多的?我