Overview of the Amazon Data Processing Agent
mainThe Amazon Data Processing Agent is a conversational AI assistant designed for AWS data engineering tasks. It uses the Model Context Protocol (MCP) to interface with the aws-dataprocessing-mcp-server, allowing users to manage AWS services via natural language.
Key Capabilities
- AWS Glue: Manage Data Catalogs (databases, tables, partitions), develop ETL jobs (PySpark/Glue Version 5), and orchestrate workflows.
- Amazon Athena: Convert natural language to SQL, optimize queries, and perform schema discovery.
- Amazon EMR-EC2: Manage clusters, handle big data workloads, and implement cost optimization (e.g., spot instances).
- S3 Operations: Manage data storage and namespaces.
Technical Stack
- Model Provider: Amazon Bedrock (Claude Sonnet 4.6, Claude 4.0 Sonnet).
- MCP Server:
aws-dataprocessing-mcp-server. - UI Framework: Streamlit with real-time streaming.
- Custom Tools:
send_email,manage_s3_table_buckets,manage_s3_namespaces,manage_s3_tables.