repository·master·Indexed 41 days ago
https://github.com/amol-/depotA Python framework for file storage and serving in web applications, providing abstractions for Local, S3, GCS, and GridFS backends. Version 0.12.0 features deep integration with SQLAlchemy and MongoDB/Ming to synchronize file lifecycles with database transactions, including automatic cleanup on rollbacks. It includes tools like DepotManager for coordination, DepotMiddleware for WSGI serving, and specialized fields like UploadedFileField for database model attachments.
DEPOT supports a variety of storage backends through a single API. Supported backends include:
depot.io.memory.MemoryFileStorage): Ideal for test suites to ensure fast tests and automatic cleanup.Depot supports multiple storage backends through various FileStorage implementations. You can choose the backend that fits your infrastructure:
depot.io.local.LocalFileStorage: Stores files on the local filesystem.depot.io.gridfs.GridFSStorage: Stores files in MongoDB using GridFS.depot.io.boto3.S3Storage or depot.io.awss3.S3Storage: Stores files in Amazon S3.depot.io.gcs.GCSStorage: Stores files in Google Cloud Storage.depot.io.memory.MemoryFileStorage: Stores files in memory (useful for testing).DEPOT supports multiple storage backends, allowing you to switch between local and cloud storage easily:
depot.io.boto3.S3Storage)Depot integrates with database sessions to manage file lifecycles. When a model object is deleted or a transaction is rolled back, any files uploaded during that unit of work or attached to the deleted objects are automatically deleted from storage.
depot.DepotExtension as a session extension.Note: Because Ming lacks a session clear entry point, files uploaded without a session flush may not be deleted when the session is removed.
To avoid breaking existing file references when migrating from one storage to another, use DepotManager.alias(alias_name, real_storage_name).
Instead of calling get() with the real storage name, always use the alias. When you want to switch the underlying storage (e.g., from local disk to GridFS), you simply update the alias to point to the new storage. The application code using the alias remains unchanged.
# Initial setup: Local storage with an alias 'avatar'
DepotManager.configure('local_avatars', {'depot.storage_path': '/var/www/lfs'})
DepotManager.alias('avatar', 'local_avatars')
# Use the alias to save files
storage = DepotManager.get('avatar')
fileid = storage.create(open('/tmp/file.png'))
# --- Later, migrate to GridFS ---
# Configure new storage
DepotManager.configure('gridfs_avatars', {
'depot.backend': 'depot.io.gridfs.GridFSStorage',
'depot.mongouri': 'mongodb://localhost/db'
})
# Update the alias to point to the new storage
DepotManager.alias('avatar', 'gridfs_avatars')
# The same code now uses GridFS automatically
storage = DepotManager.get('avatar')
fileid = storage.create(open('/tmp/file.png'))When a file is assigned to an UploadedFileField or UploadedFileProperty, it is converted into an UploadedFile object. This object is also what you receive when reloading the model from the database.
Key properties:
.file: The actual file object..url: A URL where the file can be accessed (requires DepotMiddleware in your WSGI application or storage that supports HTTP).Install the filedepot distribution using pip to use DEPOT in your Python 3.10+ applications.
$ pip install filedepotDepot allows you to attach files to database models by declaring a specific field type.
depot.fields.sqlalchemy.UploadedFileField.depot.fields.ming.UploadedFileProperty.To store a file, simply assign a file-like object (such as a cgi.FieldStorage object, bytes, or a file opened in byte mode) to the field. Depot will automatically upload the file to the default storage.
from depot.fields.sqlalchemy import UploadedFileField
class Document(Base):
__tablename__ = 'document'
uid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(16), unique=True)
content = Column(UploadedFileField)
# To store a file:
doc = Document(name=u'Foo', content=open('/tmp/document.xls', 'rb'))
DBSession.add(doc)To enable the serving of stored files via a URL, wrap your TurboGears application with the DEPOT middleware in config/middleware.py using DepotManager.make_middleware(app). This makes files accessible under the /depot URL prefix.
from depot.manager import DepotManager
app = DepotManager.make_middleware(app)To use DEPOT, you must provide a storage configuration via DepotManager.configure during your application startup. This defines where files will be physically saved. In the example below, a storage named default is configured to save files to the /tmp/ directory using the depot.storage_path key.
from depot.manager import DepotManager
DepotManager.configure('default', {'depot.storage_path': '/tmp/'})To make your stored files accessible via a URL, wrap your Flask application's WSGI application with the DEPOT middleware using DepotManager.make_middleware. This enables the serving of files under the /depot URL prefix.
from depot.manager import DepotManager
# Assuming 'app' is your Flask instance
app.wsgi_app = DepotManager.make_middleware(app.wsgi_app)You can use DEPOT in any Python script by configuring a depot via DepotManager and using its API to create and retrieve files. When retrieving a file, you receive a depot.io.interfaces.StoredFile object containing metadata like filename and content_type.
from depot.manager import DepotManager
# Configure a depot (e.g., using MongoDB GridFS)
DepotManager.configure('default', {
'depot.backend': 'depot.io.gridfs.GridFSStorage',
'depot.mongouri': 'mongodb://localhost/db'
})
depot = DepotManager.get()
# Save a file and get its unique fileid
fileid = depot.create(open('/tmp/file.png'))
# Retrieve the file object
stored_file = depot.get(fileid)
print stored_file.filename
print stored_file.content_type