aioftp

repository·master·Indexed 42 days ago

https://github.com/aio-libs/aioftp

An asynchronous FTP client and server library for Python built on top of asyncio. Version 0.28.0 provides a high-level, extensible core for FTP operations, including support for FTPS via TLS, SOCKS proxies, and throttled data transfers. It features the aioftp.Client class for standard operations like uploading, downloading, and directory management, as well as the aioftp.Server class for implementing custom FTP servers with a command dispatcher.

Tokens
16.6K
Snippets
53
Records
79
Agent score
50%

What's inside aioftp

  1. Use different path abstraction layers

    master

    The aioftp.Server uses a path_io_factory to handle file system operations. You can specify this during server initialization.

    Available implementations:

    • aioftp.PathIO: The default; performs blocking path operations.
    • aioftp.AsyncPathIO: Non-blocking operations wrapped in run_in_executor. Note: This is generally slow and should be avoided if possible.
    • aioftp.MemoryPathIO: An in-memory file system (proof of concept).
    >>> server = aioftp.Server(path_io_factory=aioftp.MemoryPathIO)
    >>> await server.start()
  2. Use the Path abstraction layer for file operations

    master

    The aioftp library provides a path abstraction layer to handle file system operations (like reading, writing, and listing) in a way that can be swapped between real file systems and in-memory storage.

    Core components include:

    • aioftp.PathIO: The standard implementation for interacting with the local file system.
    • aioftp.AsyncPathIO: An asynchronous interface for path operations.
    • aioftp.MemoryPathIO: An implementation that allows performing file operations in memory, useful for testing or transient storage without disk I/O.
    • aioftp.AbstractPathIO: The base class defining the interface for all path implementations.

    All path-related errors are raised as aioftp.PathIOError.

  3. Implement custom user management with AbstractUserManager

    master

    To control how users are authenticated and managed, you can implement or use a subclass of aioftp.AbstractUserManager.

    • aioftp.server.MemoryUserManager: A built-in implementation that stores users in memory, useful for simple or testing scenarios.
    • aioftp.User: Represents a user within the system.
    • aioftp.Permission: Defines the access rights for a user.
  4. Configure users and permissions

    master

    You can define specific users with unique credentials, home directories, and granular permissions by passing a tuple of aioftp.User objects to the aioftp.Server constructor.

    Permissions are defined using aioftp.Permission objects, which map virtual paths to readable and writable boolean flags.

    >>> users = (
    ...     aioftp.User(
    ...         "Guido",
    ...         "secret_password",
    ...         home_path="/Guido",
    ...         permissions=(
    ...             aioftp.Permission("/", readable=False, writable=False),
    ...             aioftp.Permission("/Guido", readable=True, writable=True),
    ...         )
    ...     ),
    ...     aioftp.User(
    ...         home_path="/anon",
    ...         permissions=(
    ...             aioftp.Permission("/", readable=False, writable=False),
    ...             aioftp.Permission("/anon", readable=True),
    ...         )
    ...     ),
    ... )
    >>> server = aioftp.Server(users)
    >>> await server.start()
  5. List paths on the server

    master

    Use aioftp.Client.list to list paths. It can return a list of paths or be used as an asynchronous generator for lazy iteration.

    • Eager version: await client.list() returns a list of (path, info) tuples.
    • Lazy version: async for path, info in client.list(...) allows iterating over results as they arrive.

    Warning: If you use the lazy async for version, you cannot interact with the server (e.g., issue other commands) until you have finished iterating through the loop. If you need to interact with the server while listing, use the eager version or create a second client connection.

    # Eager list (returns a list)
    paths = await client.list("/")
    
    # Lazy list (asynchronous generator)
    async for path, info in client.list("/", recursive=True):
        print(path)
    
    # Mixing lazy list with interaction (requires two clients)
    async for path, info in client1.list():
        await client2.download(path, path.name)
  6. Start a basic aioftp server

    master

    To run a minimal FTP server, create an instance of aioftp.Server and call await server.start().

    By default, the server allows anonymous login and provides read/write access to the current working directory (mapped to the virtual path /).

    >>> import aioftp
    >>> server = aioftp.Server()
    >>> await server.start()
  7. Throttle read and write speeds

    master

    You can limit data transfer speeds at multiple levels:

    • Global (Per Server): Set via read_speed_limit and write_speed_limit in the aioftp.Server constructor.
    • Per Connection: Set via read_speed_limit_per_connection and write_speed_limit_per_connection in the aioftp.Server constructor.
    • Per User: Set via read_speed_limit, write_speed_limit, etc., in the aioftp.User constructor.
    • Per User Connection: Set via read_speed_limit_per_connection and write_speed_limit_per_connection in the aioftp.User constructor.
    # Global server throttling
    >>> server = aioftp.Server(
    ...     read_speed_limit=1024 * 1024,
    ...     write_speed_limit=1024 * 1024,
    ...     read_speed_limit_per_connection=100 * 1024,
    ...     write_speed_limit_per_connection=100 * 1024
    ... )
    
    # User-specific throttling
    >>> users = (
    ...     aioftp.User(
    ...         read_speed_limit=1024 * 1024,
    ...         write_speed_limit=1024 * 1024,
    ...         read_speed_limit_per_connection=100 * 1024,
    ...         write_speed_limit_per_connection=100 * 1024
    ...     ),
    ... )
    >>> server = aioftp.Server(users)
  8. Implement Background Workers in the Server

    master

    For commands that require a passive connection (like data transfers), you should use a worker to ensure the server can continue receiving commands while data is being transferred.

    To implement this:

    1. Define an internal async function (the worker).
    2. Decorate it with @aioftp.server.worker and appropriate ConnectionConditions.
    3. In the main command method, create a task for the worker using connection.loop.create_task(coro).
    4. Add the task to connection.extra_workers so the dispatcher manages it.
    5. Send an initial response (e.g., 150) to the client immediately.
    class MyServer(aioftp.Server):
        @aioftp.ConnectionConditions(
            aioftp.ConnectionConditions.login_required,
            aioftp.ConnectionConditions.passive_server_started)
        async def coll(self, connection, rest):
    
            @aioftp.ConnectionConditions(
                aioftp.ConnectionConditions.data_connection_made,
                wait=True,
                fail_code="425",
                fail_info="Can't open data connection")
            @aioftp.server.worker
            async def coll_worker(self, connection, rest):
                stream = connection.data_connection
                del connection.data_connection
                async with stream:
                    for i in range(count):
                        binary = i.to_bytes(8, "big")
                        await stream.write(binary)
                connection.response("200", "coll transfer done")
                return True
    
            count = int(rest)
            coro = coll_worker(self, connection, rest)
            task = connection.loop.create_task(coro)
            connection.extra_workers.add(task)
            connection.response("150", "coll transfer started")
            return True
  9. Remove, rename, and manage directories

    master

    Basic filesystem management operations:

    • await client.remove(path): Removes a file or directory (recursive).
    • await client.rename(old_path, new_path): Renames or moves a file/directory.
    • await client.make_directory(path): Creates a new directory.
    • await client.change_directory(path): Changes the current working directory. Calling it without arguments returns to the root/home.
    • await client.get_current_directory(): Returns the current working directory as a PosixPath.
    # Remove a file or directory
    await client.remove("tmp.py")
    await client.remove("folder1")
    
    # Rename/Move
    await client.rename("test.py", "foo.py")
    
    # Directory management
    await client.make_directory("folder2")
    await client.change_directory("folder2")
    print(await client.get_current_directory())
    
    # Return to previous directory
    await client.change_directory()
  10. Get started with the aioftp Client

    master

    The aioftp.Client provides a high-level asynchronous API for interacting with FTP servers. You can use aioftp.Client.context() to manage the connection lifecycle automatically.

    Note on Directory Listings: For commands involving stats or listings, aioftp prefers MLSx-family commands (like MLSD) because they provide structured, machine-readable data. If the server does not support these, aioftp falls back to the LIST command, which lacks a standard format and may be harder to parse. You can provide a custom LIST parser if needed.

    import asyncio
    import aioftp
    
    
    async def get_mp3(host, port, login, password):
        async with aioftp.Client.context(host, port, login, password) as client:
            for path, info in (await client.list(recursive=True)):
                if info["type"] == "file" and path.suffix == ".mp3":
                    await client.download(path)
    
    
    async def main():
        tasks = [
            asyncio.create_task(get_mp3("server1.com", 21, "login", "password")),
            asyncio.create_task(get_mp3("server2.com", 21, "login", "password")),
            asyncio.create_task(get_mp3("server3.com", 21, "login", "password")),
        ]
        await asyncio.wait(tasks)
    
    asyncio.run(main())