SAS SWAT (SAS Cloud Analytic Services Python Interface)

repository·main·Indexed 41 days ago

https://github.com/sassoftware/python-swat

A Python interface to the SAS Cloud Analytic Services (CAS) engine, the core of the SAS Viya framework. SWAT allows developers to execute CAS analytic actions and manage large datasets using a Pythonic API that mimics Pandas, featuring objects like CASTable, CASColumn, and SASDataFrame for metadata-aware data manipulation and server-side computation.

Tokens
15.9K
Snippets
50
Records
83
Agent score
100%

What's inside SWAT

  1. Overview of SAS SWAT

    main

    The SAS SWAT package is a Python interface to SAS Cloud Analytic Services (CAS), which is the core engine of the SAS Viya framework.

    Key capabilities include:

    • Scalable Analytics: Load and analyze datasets of any size by leveraging the processing power of CAS in local or cloud environments.
    • CAS Analytic Actions: Execute complex analytic workflows directly on the CAS server.
    • Hybrid Workflows: Perform heavy lifting in CAS, then pull summarized data back to the client side for further processing in Python.
    • Pandas Compatibility: The SWAT API mimics much of the Pandas API, making it intuitive for users familiar with Python data science workflows.
    • Data Integration: Easily merge CAS-processed data with other Python data sources using Pandas data structures.
  2. License information for python-swat

    main

    The python-swat repository contains components governed by different licenses. Specifically, it includes the Apache License, Version 2.0 and the SAS License Agreement for Corrective Code or Additional Functionality.

    When using or redistributing this software, ensure you comply with the specific terms of the applicable license for the component you are using.

  3. Understand the licensing for python-swat

    main

    The python-swat package is released under a dual-license model depending on the component being used:

    1. Apache 2.0: Applies to the Python code contained in the GitHub repository.
    2. SAS License Agreement for Corrective Code or Additional Functionality: Applies to the bundled TK libraries and C extensions available on selected platforms.

    Users must ensure they are authorized under a separate SAS Software License Agreement to download and use the Code components governed by the SAS License Agreement.

  4. Understand the difference between Binary and REST protocols in SWAT

    main

    SWAT provides two ways to interact with a CAS (Cloud Analytic Services) server:

    1. Binary Protocol (Recommended):

      • Performance: Faster, especially for large data transfers.
      • Features: Offers advanced data loading and formatting.
      • Requirements: Requires pre-compiled components (included in pip installs) and the libnuma.so.1 library on Linux.
      • Platform: 64-bit Python 3.7 - 3.14 on Linux or Windows.
    2. REST Interface:

      • Performance: Slower than the binary protocol.
      • Features: Standard REST capabilities.
      • Portability: Higher portability as it uses pure Python code.
      • Requirements: Requires pandas (version 0.16 or greater).
  5. Distinguish between CAS action return values and Pandas API return values

    main

    Because SWAT blends CAS and Pandas, the return type of a method depends on whether it is a CAS action or a Pandas API method:

    1. CAS Actions: Always return a CASResults object (a subclass of a Python dictionary).
    2. Pandas API Methods: Typically return a CASTable, CASColumn, pandas.DataFrame, or pandas.Series object.

    When debugging, check the type of the output to determine if you are interacting with the CAS server or using local Pandas-style operations.

    import swat
    import os
    
    conn = swat.CAS(os.environ['CASHOST'], os.environ['CASPORT'], os.environ.get('CASUSER'), os.environ.get('CASPASSWORD'))
    tbl = conn.read_csv('https://raw.githubusercontent.com/sassoftware/sas-viya-programming/master/data/cars.csv')
    
    # CAS Action returns CASResults
    out_action = tbl.summary()
    type(out_action)  # <class 'swat.cas.results.CASResults'>
    
    # Pandas API method returns CASTable or pandas object
    out_pandas = tbl.head()
    type(out_pandas)  # <class 'swat.cas.table.CASTable'> or <class 'pandas.core.frame.DataFrame'>
  6. Handle By Group results in CASResults

    main

    When running a CAS action with By groups specified, the results are returned with specific behaviors:

    1. A result key named ByGroupInfo contains all the By group variable values.
    2. Each By group table is returned in a separate result key prefixed with ByGroup#..

    To manage these results, the swat.cas.results.CASResults class provides helper methods to select, concatenate, or filter tables across these groups.

    import swat
    import os
    
    # Setup connection
    conn = swat.CAS(os.environ['CASHOST'], os.environ['CASPORT'], os.environ.get('CASUSER'), os.environ.get('CASPASSWORD'))
    tbl = conn.read_csv('https://raw.githubusercontent.com/sassoftware/sas-viya-programming/master/data/cars.csv')
    
    # Example: Running a summary with By groups
    tbl.groupby(['Origin', 'Cylinders']).summary(subset=['Min', 'Max'])
  7. Use the CAS object to connect to a CAS server

    main

    The CAS object represents a connection to the Cloud Analytic Services (CAS) server. It is the primary entry point for executing CAS actions and includes many data reader functions similar to the Pandas package. You can use it to manage sessions, load data, and run server-side actions.

    from swat import CAS
    
    # Create a connection to the CAS server
    conn = CAS(host='your_host', port=50050, user='your_user', password='your_password')
  8. Load data from external sources like databases

    main

    CAS can connect to external sources such as databases via configured CASLibs. To load data from these sources, use the CAS.load_path method, but instead of a file path, provide the name of the resource (e.g., a database table name) within that data loader.

    # Example concept (requires configured CASLib for the database)
    # cars = conn.load_path('database_table_name', caslib='my_db_caslib')
  9. Handle CAS action results with CASResults

    main

    When a CAS action is executed, it returns a CASResults object. This object is a subclass of a Python dict, allowing you to access result objects using dictionary keys.

    CASResults also provides specialized methods for handling data that has been processed in 'By groups':

    • get_tables()
    • get_group(group_name)
    • get_set(set_name)
    • concat_bygroups()
  10. Understand the difference between SASDataFrame and pandas.DataFrame

    main

    A SASDataFrame is a subclass/extension of a pandas.DataFrame that includes SAS metadata:

    • title, label, and name for the dataframe.
    • colinfo for column-level metadata.

    Warning: When performing operations like pandas.concat on a SASDataFrame, the result may be a standard pandas.DataFrame. In this transition, all SAS-specific metadata is lost. If you need to preserve metadata, ensure your operations remain within the SWAT/SAS ecosystem.

  11. Handle CAS action outputs with CASResults

    main

    All CAS actions return a CASResults object. This object behaves like an ordered Python dictionary, allowing you to access results using standard dictionary syntax (e.g., out['key']).

    For convenience, CASResults also allows accessing keys as attributes (e.g., out.key).

    Warning: If a key name collides with a standard Python attribute or method, the attribute/method will take precedence. For production code, it is safer to use the dictionary syntax out['key'] to avoid collisions.

    Result values can be standard Python types (like dictionaries) or SASDataFrame objects (which are subclasses of pandas.DataFrame).

    # Example of accessing results
    out = conn.serverstatus()
    
    # Dictionary access (recommended for production)
    status = out['nodestatus']
    
    # Attribute access (convenient for interactive use)
    status = out.nodestatus
  12. Index CASTable columns by name, position, or mix

    main

    Indexing for CASTable objects follows a pattern similar to pandas.DataFrame. You can subset columns using names, integer positions, or a combination of both.

    Important Limitation: Because CAS tables are distributed across a grid and do not have a specified order, indexing based on a row index is not possible. To filter rows, use Boolean Indexing or the .query() method.

    Available indexing properties/methods:

    • o[columns]: Subset by column names.
    • o.loc[:, columns]: Subset by column names.
    • o.iloc[:, columns]: Subset by column integer indexes.
    • o.ix[:, columns]: Subset using a mix of names and indexes.
    • o.xs(column, axis=1): Select a cross-section (currently only supports axis=1).
    import swat
    import os
    
    # Setup connection
    conn = swat.CAS(os.environ['CASHOST'], os.environ['CASPORT'], os.environ.get('CASUSER'), os.environ.get('CASPASSWORD'))
    
    # Load a table
    tbl = conn.read_csv('https://raw.githubusercontent.com/sassoftware/sas-viya-programming/master/data/cars.csv')
    
    # Examples of different indexing styles:
    tbl['Make']              # Returns CASColumn
    tbl[['Make', 'Model']]   # Returns CASTable
    tbl.loc[:, 'Make']      # Returns CASColumn
    tbl.loc[:, 'Model':'Invoice'] # Returns CASTable (endpoints included)
    tbl.iloc[:, 1]           # Returns CASColumn
    tbl.iloc[:, 2:6]         # Returns CASTable
    tbl.ix[:, 'Model', 3]   # Mixed indexing