repository·master·Indexed 41 days ago
https://github.com/aliyun/aliyun-log-java-sdkA Java wrapper for Aliyun Log Service APIs used to programmatically manage Logstores, write log data via PutLogs, and read logs using GetLogs or Cursors. Version 0.6.97 supports Metricstore optimization, object storage within Logstores, resource tagging, and SQL query execution. For high-efficiency production environments, the Aliyun LOG Java Producer and Consumer Library are recommended over direct SDK calls.
To use the Aliyun Log Java SDK in your project, add the following dependency to your pom.xml. Note that the version in this example is 0.6.97.
<dependency>
<groupId>com.aliyun.openservices</groupId>
<artifactId>aliyun-log</artifactId>
<version>0.6.97</version>
</dependency>Add the aliyun-log dependency to your pom.xml to use the SDK.
Note on Protobuf Conflicts:
If your project encounters version conflicts with protobuf-java, use the jar-with-dependencies classifier and exclude the existing protobuf-java dependency to ensure compatibility.
<!-- Standard dependency -->
<dependency>
<groupId>com.aliyun.openservices</groupId>
<artifactId>aliyun-log</artifactId>
<version>0.6.97</version>
</dependency>
<!-- Dependency with protobuf conflict resolution -->
<dependency>
<groupId>com.aliyun.openservices</groupId>
<artifactId>aliyun-log</artifactId>
<version>0.6.97</version>
<classifier>jar-with-dependencies</classifier>
<exclusions>
<exclusion>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
</exclusion>
</exclusions>
</dependency>To interact with Aliyun Log Service, you must first instantiate a Client using your access credentials and the service endpoint.
Metricstore Optimization:
If you are sending data to a Metricstore, you can call client.setUseMetricStoreUrl(true). This is a global setting for the client that automatically appends a Hash Key (METRICS_STORE_AUTO_HASH), which can improve retrieval performance for large-cardinality time-series data. If you are writing to both Logstores and Metricstores simultaneously, you may need to manually specify the HashKey as METRICS_STORE_AUTO_HASH to trigger the automatic hash writing feature.
String accessId = "your_access_id";
String accessKey = "your_access_key";
String host = "your_endpoint";
Client client = new Client(host, accessId, accessKey);
// Optional: Enable Metricstore optimization
client.setUseMetricStoreUrl(true);The Client class is the primary entry point for interacting with Aliyun Log Service. It allows you to send requests to put or get data. While several constructors exist, it is highly recommended to use ClientBuilder (available in the SDK) to create instances.
Common ways to initialize a Client include:
CredentialsProvider (e.g., for ECS Role-based authentication).ClientConfiguration to tune timeouts and connection settings.// Basic initialization with AccessKey
Client client = new Client("your-endpoint.log.aliyuncs.com", "your-access-key-id", "your-access-key-secret");
// Initialization with custom configuration
ClientConfiguration config = new ClientConfiguration();
config.setConnectionTimeout(1000);
Client clientWithConfig = new Client("your-endpoint.log.aliyuncs.com", "accessId", "accessKey", config);The aliyun-log-java-sdk and aliyun-sls-xxx-inner libraries cannot coexist in the same project. If a transitive dependency pulls in aliyun-sls-xxx-inner, you must manually exclude it from your dependency tree.
<dependency>
<groupId>groupId1</groupId>
<artifactId>artifactId1</artifactId>
<version>version1</version>
<exclusions>
<exclusion>
<groupId>com.aliyun.openservices</groupId>
<artifactId>aliyun-sls-xxx-inner</artifactId>
</exclusion>
</exclusions>
</dependency>The aliyun-log-java-sdk and aliyun-sls-xxx-inner packages cannot coexist in the same project. If a dependency you are using pulls in aliyun-sls-xxx-inner, you must manually exclude it from that dependency's configuration.
<dependency>
<groupId>groupId1</groupId>
<artifactId>artifactId1</artifactId>
<version>version1</version>
<exclusions>
<exclusion>
<groupId>com.aliyun.openservices</groupId>
<artifactId>aliyun-sls-xxx-inner</artifactId>
</exclusion>
</exclusions>
</dependency>If you encounter Protobuf version conflicts, you can use the jar-with-dependencies classifier provided by the SDK. This version includes the necessary dependencies bundled within the JAR. You should also exclude the protobuf-java dependency to prevent classpath issues.
<dependency>
<groupId>com.aliyun.openservices</groupId>
<artifactId>aliyun-log</artifactId>
<version>0.6.97</version>
<classifier>jar-with-dependencies</classifier>
<exclusions>
<exclusion>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
</exclusion>
</exclusions>
</dependency>Logs are sent in groups using the PutLogs method. Each group contains LogItem objects. Each LogItem represents a single log entry with key-value pairs.
Best Practice Note: For high-efficiency production environments, it is recommended to use the Aliyun LOG Java Producer instead of calling the SDK directly to improve IO efficiency.
int numLogGroup = 10;
for (int i = 0; i < numLogGroup; i++) {
List<LogItem> logGroup = new ArrayList<LogItem>();
// Create first log entry
LogItem logItem = new LogItem((int) (new Date().getTime() / 1000));
logItem.PushBack("level", "info");
logItem.PushBack("name", String.valueOf(i));
logItem.PushBack("message", "it's a test message");
logGroup.add(logItem);
// Create second log entry
LogItem logItem2 = new LogItem((int) (new Date().getTime() / 1000));
logItem2.PushBack("level", "error");
logItem2.PushBack("name", String.valueOf(i));
logItem2.PushBack("message", "it's a test message");
logGroup.add(logItem2);
try {
client.PutLogs(project, logStore, topic, logGroup, "");
} catch (LogException e) {
System.out.println("error code :" + e.GetErrorCode());
System.out.println("error message :" + e.GetErrorMessage());
System.out.println("error requestId :" + e.GetRequestId());
throw e;
}
}Use the Client.CreateLogStore method to create a new Logstore within a specific project. You must define the LogStore object with the desired name, Time-To-Live (TTL) in days, and the number of shards.
String project = "your_project_name";
String logstore = "your_logstore";
int ttl_in_day = 3;
int shard_count = 10;
LogStore store = new LogStore(logstore, ttl_in_day, shard_count);
CreateLogStoreResponse res = client.CreateLogStore(project, store);Initialize a Client using your Aliyun AccessKey ID, AccessKey, and the service endpoint host.
Metricstore Optimization:
If you are writing to a Metricstore (rather than a standard Logstore), call client.setUseMetricStoreUrl(true). This is a global setting at the Client level that enables auto-appending the hash key (METRICS_STORE_AUTO_HASH), which improves query performance for high-cardinality time-series data.
String accessId = "your_access_id";
String accessKey = "your_access_key";
String host = "your_endpoint";
Client client = new Client(host, accessId, accessKey);
// Enable auto-hash for Metricstore destinations
client.setUseMetricStoreUrl(true);To consume logs, you first obtain a cursor using GetCursor for a specific shard and time range. Then, you use BatchGetLog in a loop to fetch batches of logs. The loop continues until the nextCursor returned by the response is equal to the current cursor, indicating no more data is available.
Best Practice Note: For production log consumption, it is recommended to use the Consumer Library which provides advanced features like load balancing and ordered consumption.
int shardId = 0;
GetCursorResponse res;
try {
// Get cursor for logs received in the last hour
long fromTime = (int)(System.currentTimeMillis()/1000.0 - 3600);
res = client.GetCursor(project, logStore, shardId, fromTime);
System.out.println("shard_id:" + shardId + " Cursor:" + res.GetCursor());
} catch (LogException e) {
e.printStackTrace();
}
String cursor = res.GetCursor();
while(true) {
BatchGetLogResponse logDataRes = client.BatchGetLog(project, logStore, shardId, 100, cursor);
List<LogGroupData> logGroups = logDataRes.GetLogGroups();
String nextCursor = logDataRes.GetNextCursor();
System.out.print("The Next cursor:" + nextCursor);
if (cursor.equals(nextCursor)) {
break;
}
cursor = nextCursor;
}To consume logs, follow a two-step process:
client.GetCursor specifying the project, logstore, shard ID, and a starting timestamp (in seconds).client.BatchGetLog with the retrieved cursor to fetch logs.Continue reading by using the nextCursor returned in the BatchGetLogResponse. If nextCursor is equal to the current cursor, you have reached the end of the available data.
Important Consumption Note: For production consumption, do not use these raw pull APIs directly. Use the Aliyun Consumer Library which provides load balancing and ordered consumption.
int shardId = 0;
GetCursorResponse res;
try {
// Cursor for logs from the last hour
long fromTime = (int)(System.currentTimeMillis()/1000.0 - 3600);
res = client.GetCursor(project, logStore, shardId, fromTime);
System.out.println("shard_id:" + shardId + " Cursor:" + res.GetCursor());
} catch (LogException e) {
e.printStackTrace();
}
String cursor = res.GetCursor();
while(true) {
BatchGetLogResponse logDataRes = client.BatchGetLog(project, logStore, shardId, 100, cursor);
List<LogGroupData> logGroups = logDataRes.GetLogGroups();
String nextCursor = logDataRes.GetNextCursor();
System.out.print("The Next cursor:" + nextCursor);
if (cursor.equals(nextCursor)) {
break;
}
cursor = nextCursor;
}