amazon web服务使用Java SDK从AWS EMR集群获取主公共DNS值

Question

我需要通过Java SDK获取主公共DNS值。在应用程序开始时，我将获得的唯一信息是ClusterName，它是静态的

到目前为止，我已经能够提取出我需要的所有其他信息，除了这个和这个，不幸的是，对于应用程序的成功至关重要

这是我目前正在使用的代码：

List<ClusterSummary> summaries = clusters.getClusters();
        for (ClusterSummary cs: summaries) {
            if (cs.getName().equals("test") && WHITELIST.contains(cs.getStatus().getState())) {
                ListInstancesResult instances = emr.listInstances(new ListInstancesRequest().withClusterId(cs.getId()));
                clusterHostName = instances.getInstances().get(0).toString();
                jobFlowId = cs.getId();
            }
        }

我已经删除了get for PublicIpAddress，因为我想测试完整的toString。我应该清楚的是，这个方法确实提供了我需要的DNS，但我无法区分它们

如果我的EMR有4台机器，我不知道该实例将位于列表中的哪个位置。对于我的基本测试，我只有两台机器，一台主机和一台工人.get(0)已在连续运行中返回了master和worker的值

下面是我能从中获得的信息——目前我唯一能看到的选择是使用“ReadyDateTime”作为标识符，因为主数据“应该”总是先准备好的，但这让人感觉很不舒服，我希望有一个更干净的解决方案

    {Id: id,
Ec2InstanceId: id,
PublicDnsName: ec2-54--143.compute-1.amazonaws.com,
PublicIpAddress: 54..143,
PrivateDnsName: ip-10--158.ec2.internal,
PrivateIpAddress: 10..158,
Status: {State: RUNNING,StateChangeReason: {},
Timeline: {CreationDateTime: Tue Feb 21 09:18:08 GMT 2017,
ReadyDateTime: Tue Feb 21 09:25:11 GMT 2017,}},
InstanceGroupId: id,
EbsVolumes: []}

{Id: id,
Ec2InstanceId: id,
PublicDnsName: ec2-54--33.compute-1.amazonaws.com,
PublicIpAddress: 54..33,
PrivateDnsName: ip-10--95.ec2.internal,
PrivateIpAddress: 10..95,
Status: {State: RUNNING,StateChangeReason: {},
Timeline: {CreationDateTime: Tue Feb 21 09:18:08 GMT 2017,
ReadyDateTime: Tue Feb 21 09:22:48 GMT 2017,}},
InstanceGroupId: id
EbsVolumes: []}

Answer 1

不要使用ListInstances。相反，使用DescribeCluster，它作为MasterPublicDnsName字段之一返回

Answer 2

 AWSCredentials credentials_profile = null;
 credentials_profile = new 
 DefaultAWSCredentialsProviderChain().getCredentials();

 AmazonElasticMapReduceClient emr = new 
 AmazonElasticMapReduceClient(credentials_profile);
    Region euWest1 = Region.getRegion(Regions.US_EAST_1);
    emr.setRegion(euWest1);
 DescribeClusterFunction fun = new DescribeClusterFunction(emr);
    DescribeClusterResult res = fun.apply(new 
 DescribeClusterRequest().withClusterId(clusterId));
    String publicDNSName =res.getCluster().getMasterPublicDnsName();

下面是获取公共DNS名称的工作代码

Answer 3

进一步阐述乔纳森所提到的内容：

        AmazonEC2Client ec2 = new AmazonEC2Client(cred);
    DescribeInstancesResult describeInstancesResult = ec2.describeInstances(new DescribeInstancesRequest().withInstanceIds(clusterInstanceIds));
    List<Reservation> reservations = describeInstancesResult.getReservations();
    for (Reservation res : reservations) {
        for (GroupIdentifier group : res.getGroups()) {
            if (group.getGroupName().equals("ElasticMapReduce-master")) { // yaaaaaaaaah, Wahay!
                masterDNS = res.getInstances().get(0).getPublicDnsName();
            }
        }
    }