aws php sdk s3 实例,aws sdk cpp中S3相关的应用和示例
概述
aws提供多种sdk去访问S3，包括java、go、php、js、ruby、net、c++等，本篇文章结合作者最近应用的实践，介绍aws sdk cpp中访问S3的使用，示例中包括对S3的基本put，get等操作。
在使用aws sdk cpp中，发现它还不是那么完善，很多对S3的操作都找不到示例，并且github上这部分的源码更新很快，估计也是在持续开发和完善中，建议读者在自己应用时，下载最新的sdk版本后测试。
安装
测试系统：CentOS Linux release 7.2.1511
下载github上的sdk源码
1# git clone https://github.com/aws/aws-sdk-cpp
或者直接下载zip压缩包后解压缩
安装依赖包
1
2
3
4
5yum install -y gcc-c++
yum install -y cmake
yum install -y zlib-devel
yum install -y openssl-devel
yum install -y curl-devel
Ubuntu系统上：
1
2
3
4
5
6apt-get install -y g++
apt-get install -y cmake
apt-get install -y libssl-dev
apt-get install -y libcurl4-openssl-dev
apt-get install -y uuid-dev
apt-get install -y libboost-all-dev
安装aws cpp sdk
1
2
3
4
5
6
7
8
9mkdir build_dir
cd build_dir
cmake -DCMAKE_BUILD_TYPE=Release ../aws-sdk-cpp
# just make and install core and s3
make -j `nproc` -C aws-cpp-sdk-core
make -j `nproc` -C aws-cpp-sdk-s3
make install -C aws-cpp-sdk-core
make install -C aws-cpp-sdk-s3
示例
基本代码
要使用aws c++ sdk，必须包含如下基本代码
1
2
3
4
5
6
7
8
9
10
11#include
int main(int argc, char** argv)
{
Aws::SDKOptions options;
Aws::InitAPI(options);
{
// make your SDK calls here.
}
Aws::ShutdownAPI(options);
return 0;
}
使用aws s3 client来访问对象存储
1
2
3
4
5
6
7
8
9
10Aws::Client::ClientConfiguration cfg;
cfg.endpointOverride = "your s3 endpoint";
cfg.scheme = Aws::Http::Scheme::HTTP;
cfg.connectTimeoutMs = 100000;
cfg.requestTimeoutMs = 100000;
Aws::Auth::AWSCredentials cred("your access key", "your secret key");
S3Client client(cred, cfg, false, false);
// Then you can use this client object to access s3 object
示例1 - list bucket1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38cat aws-s3-bucket.cpp
#include
#include
#include
#include
using namespace Aws::S3;
using namespace Aws::S3::Model;
using namespace std;
int main(int argc, char* argv[])
{
Aws::SDKOptions options;
Aws::InitAPI(options);
Aws::Client::ClientConfiguration cfg;
cfg.endpointOverride = "...";
cfg.scheme = Aws::Http::Scheme::HTTP;
Aws::Auth::AWSCredentials cred("...", "...");
S3Client client(cred, cfg, false, false);
auto response = client.ListBuckets();
if (response.IsSuccess()) {
auto buckets = response.GetResult().GetBuckets();
for (auto iter = buckets.begin(); iter != buckets.end(); ++iter) {
cout << iter->GetName() << "\t"
<< iter->GetCreationDate().ToLocalTimeString(Aws::Utils::DateFormat::ISO_8601) << endl;
}
} else {
cout << "Error while ListBuckets " << response.GetError().GetExceptionName()
<< " " << response.GetError().GetMessage() << endl;
}
Aws::ShutdownAPI(options);
return 0;
}
示例2 - get object1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33/*
* Get object range content to local file
*/
int S3Storage::get_object_to_file(const S3Client &client,
const Aws::String bucket,
const Aws::String object,
const string &file_name,
const uint64_t offset,
uint64_t length)
{
...
GetObjectRequest request;
request.WithBucket(bucket).WithKey(object);
string range = S3_RANGE_STRING(offset, length);
request.SetRange(range.c_str());
auto outcome = client.GetObject(request);
if(!outcome.IsSuccess()) {
cout << "GetObject error: " <<
outcome.GetError().GetExceptionName() << " - " <<
outcome.GetError().GetMessage() << endl;
return -1;
}
Aws::OFStream local_file;
local_file.open(file_name.c_str(), std::ios::in | std::ios::out | std::ios::binary);
assert(local_file.good());
local_file.seekp(offset, std::ios::beg);
local_file << outcome.GetResult().GetBody().rdbuf();
local_file.close();
...
}
示例2 - put object1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28/*
* Put whole file to s3 object
*/
int S3Storage::put_object_from_file(const S3Client &client,
const Aws::String bucket,
const Aws::String object,
const string &file_name)
{
...
PutObjectRequest request;
request.WithKey(object).WithBucket(bucket);
auto input_data = Aws::MakeShared<:fstream>("PutObjectInputStream",
file_name.c_str(), std::ios_base::in | std::ios_base::binary);
//set the stream that will be put to s3
request.SetBody(input_data);
auto outcome = client.PutObject(request);
if(outcome.IsSuccess()) {
cout << "Put object succeeded!" << endl;
} else {
cout << "PutObject error: " <<
outcome.GetError().GetExceptionName() << " - " <<
outcome.GetError().GetMessage() << endl;
}
...
}
因有需要读取文件的一部分内容，然后上传到S3上的一个object，查询了好多资料没找到示例，最后翻阅aws cpp sdk的源码，参考相关的实现，花费了很多时间才试验出来，贴给有相同需要的朋友。。。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42/*
* Put one file slice to s3 object
*/
int S3Storage::put_object_from_file_slice(const S3Client &client,
const Aws::String bucket,
const Aws::String object,
const string &file_name,
const uint64_t offset,
const uint64_t length)
{
...
// Upload file with slices to s3 paralleled
PutObjectRequest request;
request.WithKey(object).WithBucket(bucket);
Aws::FStream local_file;
local_file.open(file_name.c_str(), std::ios::in | std::ios::binary);
assert(local_file.good());
Array file_contents(length);
local_file.seekg(offset, std::ios::beg);
cout << "file read offset " << local_file.tellg() << endl;
local_file.read((char*)file_contents.GetUnderlyingData(),
file_contents.GetLength());
local_file.close();
//set the stream that will be put to s3
auto sbuff = Aws::New<:utils::stream::preallocatedstreambuf>(CLASS_TAG,
&file_contents, file_contents.GetLength());
auto sbody = Aws::MakeShared<:iostream>("whatever string", sbuff);
request.SetBody(sbody);
auto outcome = client.PutObject(request);
if(outcome.IsSuccess()) {
cout << "Put object succeeded!" << endl;
} else {
cout << "PutObject error: " <<
outcome.GetError().GetExceptionName() << " - " <<
outcome.GetError().GetMessage() << endl;
}
...
}
参考