初始化

  1. 确定Endpoint

目前有效的Endpoint为:nos-eastchina1.126.net

  1. 获取密钥对

使用AWS S3 Java SDK前,您需要拥有一个有效的 Access Key(包括Access Key和Access Secret)用来进行签名认证。可以通过如下步骤获得:

1)登录 https://c.163.com/ 注册用户

2)注册后,蜂巢会颁发 Access Key 和 Secret Key 给客户,您可以在蜂巢“用户中心”的“Access Key”查看并管理您的Access Key

  1. 在代码中实例化AmazonS3

S3推荐获取accessKey,secretKey的方式

AWSCredentials credentials = null;
try {
    credentials = new ProfileCredentialsProvider().getCredentials();
} catch (Exception e) {
    throw new AmazonClientException(
            "Cannot load the credentials from the credential profiles file. " +
                    "Please make sure that your credentials file is at the correct " +
                    "location (~/.aws/credentials), and is in valid format.",
            e);
}

  /*
  credentials文件的目录:~/.aws/credentials
  credentials文件中的格式:
  [default]
  aws_access_key_id = xxxx
  aws_secret_access_key = xxx
  */

推荐使用的方式

String accessKey = "your-accesskey";
String secretKey = "your-secretKey ";
AWSCredentials credentials = new BasicAWSCredentials(accessKey,secretKey);
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
            .withCredentials(new AWSStaticCredentialsProvider(credentials))
            .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("endpoint","region"))
            .build();//endpoint,region请指定为NOS支持的(us-east-1对应hz,us-east2对应bj)

不推荐使用的方式

String accessKey = "your-accesskey";
String secretKey = "your-secretKey ";
AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
AmazonS3 s3Client = new AmazonS3Client(credentials);//该方法弃用,不推荐使用
s3Client.setEndpoint(endPoint);//由于s3的sdk默认使用的是s3的服务地址,所以如果要使用NOS的服务,必须显示指定Endpoint

Note

注:AmazonS3是线程安全的,可以并发使用

  1. 配置AmazonS3

如果您需要修改AmazonS3的默认参数,可以在实例化AmazonS3时传入ClientConfiguration实例。ClientConfiguration是AmazonS3的配置类,可配置连接超时、最大连接数等参数。通过ClientConfiguration可以设置的参数见下表:

参数描述调用方法
connectionTimeout建立连接的超时时间(单位:毫秒)默认:50000毫秒setConnectionTimeout
maxConnections允许打开的最大HTTP连接数默认:50setMaxConnections
socketTimeoutSocket层传输数据超时时间(单位:毫秒)默认:50000毫秒setSocketTimeout
maxErrorRetry请求失败后最大的重试次数默认:3次setMaxErrorRetry
protocol使用http协议还是https协议默认:https协议setProtocol

带ClientConfiguration参数实例化AmazonS3的示例代码:

import com.amazonaws.ClientConfiguration;
import com.amazonaws.Protocol;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

String accessKey = "your-accesskey";
String secretKey = "your-secretKey ";
AWSCredentials credentials = new BasicAWSCredentials(accessKey,secretKey);
ClientConfiguration conf = new ClientConfiguration();
// 设置AmazonS3使用的最大连接数
conf.setMaxConnections(200);
// 设置socket超时时间
conf.setSocketTimeout(10000);
// 设置失败请求重试次数
conf.setMaxErrorRetry(2);
// 如果要用https协议,请加上下面语句
conf.setProtocol(Protocol.HTTPS);

//AmazonS3 s3Client = new AmazonS3Client(credentials,clientConfiguration);
//s3Client.setEndpoint(endPoint);
  AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
	            .withCredentials(new AWSStaticCredentialsProvider(credentials))
	            .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("endpoint","region"))
            .withClientConfiguration(conf)
            .build();//endpoint,region请指定为NOS支持的

Note

注意:后面的示例代码默认您已经实例化了所需的AmazonS3对象

  1. 在代码中实例化AmazonS3Encryption

可以把AmazonS3Encryption当做AmazonS3用,除了StrictAuthenticatedEncryption这个不支持range,且要求JDK必须在1.7及以上,由于NOS目前不支持KMS,所以用户的CMK只能自己管理

String accessKey = "your-accesskey";
String secretKey = "your-secretKey ";
//1.获取CMK,客户端主密钥,可以使用对称和分对称两种方式,下述使用的是非对称的
KeyPair keyPair = loadKeyPair(keyDir,"RSA");

// 2. Construct an instance of AmazonS3Encryption.
EncryptionMaterials encryptionMaterials = new EncryptionMaterials(
        keyPair);
ClientConfiguration configuration = new ClientConfiguration();
configuration.setProtocol(Protocol.HTTPS);
CryptoConfiguration cryptoConfiguration = new CryptoConfiguration();
//支持EncryptionOnly,AuthenticatedEncryption,StrictAuthenticatedEncryption,默认是EncryptionOnly,StrictAuthenticatedEncryption不支持range请求
cryptoConfiguration.setCryptoMode(CryptoMode.StrictAuthenticatedEncryption);
//保存加密信息的方式,有两种方式,Instruction模式和Metadata模式,由于NOS分块上传和S3支持上存在一些差异,导致metadata保存的方式大文件下载时由于找不到加密信息而不解密
cryptoConfiguration.setStorageMode(CryptoStorageMode.InstructionFile);

EncryptionMaterialsProvider encryptionMaterialsProvider = new StaticEncryptionMaterialsProvider(encryptionMaterials);


ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setProtocol(Protocol.HTTPS);
        AwsClientBuilder.EndpointConfiguration endpointConfiguration = new AwsClientBuilder.EndpointConfiguration("nos-eastchina1.126.net","us-east-1");
AmazonS3Encryption encryptionClient = AmazonS3EncryptionClientBuilder.standard().withCryptoConfiguration(cryptoConfiguration).
        withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey,secretKey)))
        .withEncryptionMaterials(encryptionMaterialsProvider).withClientConfiguration(clientConfiguration).withEndpointConfiguration(endpointConfiguration).build();


//loadKeyPair的实现方式,非对称加密algorithm = RSA
KeyPair loadKeyPair(String path, String algorithm)
        throws IOException, NoSuchAlgorithmException,
        InvalidKeySpecException {
    // read public key from file
    File filePublicKey = new File(path + "/public.key");
    FileInputStream fis = new FileInputStream(filePublicKey);
    byte[] encodedPublicKey = new byte[(int) filePublicKey.length()];
    fis.read(encodedPublicKey);
    fis.close();

    // read private key from file
    File filePrivateKey = new File(path + "/private.key");
    fis = new FileInputStream(filePrivateKey);
    byte[] encodedPrivateKey = new byte[(int) filePrivateKey.length()];
    fis.read(encodedPrivateKey);
    fis.close();

    // Convert them into KeyPair
    KeyFactory keyFactory = KeyFactory.getInstance(algorithm);
    X509EncodedKeySpec publicKeySpec = new X509EncodedKeySpec(
            encodedPublicKey);
    PublicKey publicKey = keyFactory.generatePublic(publicKeySpec);

    PKCS8EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(
            encodedPrivateKey);
    PrivateKey privateKey = keyFactory.generatePrivate(privateKeySpec);

    return new KeyPair(publicKey, privateKey);
}
//对称加密,algorithm = AES
SecretKey loadKeyPair(String path, String algorithm)
        throws IOException {
    //Read private key from file.
    File keyFile = new File(path);
    FileInputStream keyfis = new FileInputStream(keyFile);
    byte[] encodedPrivateKey = new byte[(int) keyFile.length()];
    keyfis.read(encodedPrivateKey);
    keyfis.close();
    //Generate secret key.
    return new SecretKeySpec(encodedPrivateKey, algorithm);
}

//生成Key的方式,非对称加密
public static KeyPair genKeyPair(String algorithm, int bitLength)
        throws NoSuchAlgorithmException {
    KeyPairGenerator keyGenerator = KeyPairGenerator.getInstance(algorithm);
    SecureRandom srand = new SecureRandom();
    keyGenerator.initialize(bitLength, srand);
    return keyGenerator.generateKeyPair();
}
//生成Key的方式,对称加密
SecretKey generateCMasterKey() throws IOException, InvalidKeySpecException, NoSuchAlgorithmException {
    KeyGenerator symKeyGenerator = null;
    try {
        symKeyGenerator = KeyGenerator.getInstance("AES");
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    }
    symKeyGenerator.init(256);
    return  symKeyGenerator.generateKey();
}
//保存Key,该key只要生成一次就好了,要妥善保管,如果该key丢失了,那么意味着通过该key加密的数据将没法解密
//非对称
public static void saveKeyPair(String dir, KeyPair keyPair)
        throws IOException {
    PrivateKey privateKey = keyPair.getPrivate();
    PublicKey publicKey = keyPair.getPublic();

    X509EncodedKeySpec x509EncodedKeySpec = new X509EncodedKeySpec(
            publicKey.getEncoded());
    FileOutputStream fos = new FileOutputStream(dir + "/public.key");
    fos.write(x509EncodedKeySpec.getEncoded());
    fos.close();

    PKCS8EncodedKeySpec pkcs8EncodedKeySpec = new PKCS8EncodedKeySpec(
            privateKey.getEncoded());
    fos = new FileOutputStream(dir + "/private.key");
    fos.write(pkcs8EncodedKeySpec.getEncoded());
    fos.close();
}
//对称
void saveSymmetricKey(String path, SecretKey secretKey)
        throws IOException {
    X509EncodedKeySpec x509EncodedKeySpec = new X509EncodedKeySpec(
            secretKey.getEncoded());
    FileOutputStream keyfos = new FileOutputStream(path);
    keyfos.write(x509EncodedKeySpec.getEncoded());
    keyfos.close();
}

加密模式

Java SDKCryptoModeEncryptDecryptRange GetMultipart Upload
1.7.8.1+AEAES‑GCMAES‑GCMYesYES
1.7.8.1+SAEAES‑GCMAES‑GCMNoYES
1.7.8.1+EOAES‑CBCAES‑GCMAES-CBCYES

CryptoMode

  • AE : AuthenticatedEncryption
  • SAE : StrictAuthenticatedEncryption
  • EO : EncryptionOnly

桶管理

创建桶

您可以通过AmazonS3.createBucket创建一个桶。示例代码如下:

//设置您要创建桶的名称
CreateBucketRequest request = new CreateBucketRequest(bucketName);
//设置桶的权限,如果不设置,默认为Private
request.setCannedAcl(CannedAccessControlList.PublicRead);
AmazonS3.createBucket(request);

Attention

注意:

1.桶的命名规范参见API文档

2.NOS中的桶名是全局唯一的,您或者他人已经创建了同名桶,您无法再创建该名称的桶

列举桶

您可以通过AmazonS3.listBuckets列举出当前用户拥有的所有桶。示例代码如下:

for (Bucket bucket : s3Client.listBuckets()) {
     System.out.println(" - " + bucket.getName());
}

删除桶

您可以通过AmazonS3.deleteBucket删除指定的桶。示例代码如下:

s3Client.deleteBucket(bucketName);

Attention

注意:

如果指定的桶不为空(桶中有文件或者未完成的分块上传),则桶无法删除

查看桶是否存在

您可以通过AmazonS3.doesBucketExist查看指定的桶是否存在。示例代码如下:

boolean exists = s3Client.doesBucketExist(bucketName);

Attention

注意:

您或者他人已经创建了指定名称的桶,doesBucketExist都会返回true。否则返回false*

设置桶的ACL

桶的ACL包含两类:Private(私有), PublicRead(公共读私有写)。您可以通过AmazonS3.setBucketAcl设置桶的权限。

权限SDK中的对应值
私有读写,对应的Permission为WRITECannedAccessControlList.Private
公共读私有写,对应的Permission为READCannedAccessControlList.PublicRead
公共读写CannedAccessControlList.PublicReadWrite,NOS不支持

示例代码如下:

//method 1,使用header的方式
AmazonS3.setBucketAcl(bucketName, CannedAccessControlList.Private);

//method 2,使用body的方式
AccessControlList accessControlList1 = new AccessControlList();
Grantee grantee = new CanonicalGrantee("aa");
accessControlList1.setOwner(new Owner());
accessControlList1.grantPermission(grantee,Permission.Write);//授权列表中的第一个值为有效值,其他的值会被忽略
s3Client.setBucketAcl(bucketName,accessControlList1);

查看桶的ACL

您可以通过AmazonS3.getBucketAcl()查看桶的权限。示例代码如下:

AccessControlList accessControlList = s3Client.getBucketAcl(bucketName);
System.out.println("owner : " +  accessControlList.getOwner().getId() + " : " + accessControlList.getOwner().getDisplayName());
for(Grant grant : accessControlList.getGrantsAsList()){//NOS由于权限不能赋值给其他的用户,所以返回值中只有一条记录
      System.out.println(grant.getGrantee().getIdentifier() + " : " +  grant.getPermission() + " : " + grant.getGrantee().getTypeIdentifier());
}