stacks-storage

# Stacks Storage File system abstraction with a Laravel-style Storage facade, local/S3/Azure adapters, file upload handling, and low-level file utilities. ## Key Paths - Core package: `storage/framework/core/storage/src/` - Storage facade: `storage/framework/core/storage/src/facade.ts` - Uploaded file: `storage/framework/core/storage/src/uploaded-file.ts` - Types: `storage/framework/core/storage/src/types.ts` - Filesystem config types: `storage/framework/core/storage/src/types/filesystem.ts` - Local adapter: `storage/framework/core/storage/src/adapters/local.ts` - S3 adapter: `storage/framework/core/storage/src/adapters/s3.ts` - Azure adapter: `storage/framework/core/storage/src/adapters/azure.ts` - Azure request signing: `storage/framework/core/storage/src/azure-signing.ts` - Memory adapter: `storage/framework/core/storage/src/adapters/memory.ts` - Bun adapter: `storage/framework/core/storage/src/adapters/bun.ts` - File utilities: `storage/framework/core/storage/src/files.ts` - Folder utilities: `storage/framework/core/storage/src/folders.ts` - Copy: `storage/framework/core/storage/src/copy.ts` - Move: `storage/framework/core/storage/src/move.ts` - Delete: `storage/framework/core/storage/src/delete.ts` - Hash: `storage/framework/core/storage/src/hash.ts` - Glob: `storage/framework/core/storage/src/glob.ts` - Zip: `storage/framework/core/storage/src/zip.ts` - Helpers: `storage/framework/core/storage/src/helpers.ts` - Configuration: `config/filesystems.ts` ## Package Exports ```typescript // Storage facade (Laravel-style) import { Storage, StorageManager } from '@stacksjs/storage' // File uploads import { UploadedFile, uploadedFile, uploadedFiles } from '@stacksjs/storage' // Adapters import { createLocalStorage, LocalStorageAdapter } from '@stacksjs/storage' import { createS3Storage, S3StorageAdapter } from '@stacksjs/storage' // Config helpers import { localDisk, s3Disk, azureDisk, r2Disk, gcsDisk, filebaseDisk, backblazeDisk, hetznerDisk, configFromEnv } from '@stacksjs/storage' // Types import type { StorageAdapter, FileContents, StatEntry, DirectoryEntry, DirectoryListing } from '@stacksjs/storage' import type { ListOptions, PublicUrlOptions, TemporaryUrlOptions, ChecksumOptions, MimeTypeOptions } from '@stacksjs/storage' import type { DiskConfig, FilesystemConfig, LocalDiskConfig, S3DiskConfig } from '@stacksjs/storage' import { Visibility, createDirectoryListing, normalizeExpiryToMilliseconds, normalizeExpiryToDate, isFile, isDirectory } from '@stacksjs/storage' // Low-level file operations import { copy, copyFile, copyFolder } from '@stacksjs/storage' import { storage } from '@stacksjs/storage' // storage.move, storage.rename import { del, deleteFile, deleteFolder, deleteEmptyFolder, deleteEmptyFolders, deleteGlob, isDirectoryEmpty } from '@stacksjs/storage' import { readJsonFile, readPackageJson, readTextFile, writeFile, writeJsonFile, writeTextFile, put, get, getFiles, deleteFiles, hasFiles } from '@stacksjs/storage' import { isFolder, isDir, doesFolderExist, createFolder, getFolders } from '@stacksjs/storage' import { glob, globSync } from '@stacksjs/storage' import { hashDirectory, hashPath, hashPaths } from '@stacksjs/storage' import { zip, unzip, archive, unarchive, compress, decompress, gzipSync, gunzipSync, deflateSync, inflateSync, zipEntries } from '@stacksjs/storage' import type { ZipEntriesOptions, ZipEntry } from '@stacksjs/storage' import * as storage from '@stacksjs/storage' ``` ## Storage Facade (StorageManager) ### Recoverable temporary workspaces Use `TemporaryWorkspaceJournal` for workers that recover temporary directories after a process interruption. Keep its private journal outside job-controlled checkouts and separate it by stable worker and controller identity. `create(key)` returns a frozen live ownership handle; stop its resources, then call `complete(handle)` for normal disposal. A restarted instance cannot forge this capability from a listed record. Both live completion and recovered retirement recheck the current mount table immediately before deletion. Nested mounts, including bind mounts on the same device, preserve the workspace and journal. Incomplete mount inspection also preserves them: recursive deletion can unlink mounted data before reporting that the mount point is busy. Crash recovery uses `inspectRetirement(id, mayRetire)` for a read-only preview and `retire(id, mayRetire)` to apply the same checks. Both require an authoritative lifecycle decision and preserve changed, mounted or in-use directories. Age alone never authorizes deletion. `inspectTemporaryWorkspace` reports unavailable process visibility explicitly. Protected same-user peers can require privileged maintenance on Linux; never ignore permission errors. An already-privileged operator may set `ownerUid` to maintain an existing private journal belonging to that user. This does not create another user's workspaces or grant new privileges. Unsupported or incomplete inspection always preserves data. ```ts import { TemporaryWorkspaceJournal } from '@stacksjs/storage' const journal = new TemporaryWorkspaceJournal({ directory: privateJournalDirectory, temporaryRoot, prefix: 'worker-job-' }) const workspace = journal.create(jobId) // Stop the live workspace's resources before normal disposal. journal.complete(workspace) // After restart, ask the controller about a recovered record. await journal.retire(recovered.id, record => controllerConfirmsFinished(record.key)) ``` The `Storage` singleton is a pre-instantiated `StorageManager`. It lazily builds its config from `@stacksjs/config` (which reads `config/filesystems.ts` and env vars). ### Basic Operations ```typescript import { Storage } from '@stacksjs/storage' // Write a file (to default disk) await Storage.put('file.txt', 'Hello World') await Storage.put('data.bin', new Uint8Array([1, 2, 3])) // Read a file as string const content = await Storage.get('file.txt') // Check existence const exists = await Storage.exists('file.txt') const missing = await Storage.missing('file.txt') // Delete await Storage.delete('file.txt') // Copy and move await Storage.copy('source.txt', 'dest.txt') await Storage.move('old.txt', 'new.txt') // File info const size = await Storage.size('file.txt') // bytes const modified = await Storage.lastModified('file.txt') // ms timestamp const mime = await Storage.mimeType('file.txt') const hash = await Storage.checksum('file.txt', 'sha256') // 'md5' | 'sha1' | 'sha256' // Public URL const url = await Storage.url('file.txt') // Directories await Storage.makeDirectory('uploads/images') await Storage.deleteDirectory('uploads/old') // List files (returns AsyncIterable<{ path, type }>) for await (const entry of Storage.files('uploads/')) { console.log(entry.path, entry.type) // 'file' or 'directory' } // Recursive listing for await (const entry of Storage.allFiles('uploads/')) { console.log(entry.path) } ``` ### Using Named Disks ```typescript // Use a specific disk await Storage.disk('s3').write('uploads/file.txt', contents) await Storage.disk('public').write('images/logo.png', imageData) await Storage.disk('local').readToString('config.json') // Disk instances are cached -- subsequent calls return the same adapter ``` ### Configuring Disks ```typescript // Initialize with custom config (overrides env-based config) Storage.init({ default: 's3', disks: { custom: { driver: 'local', root: '/custom/path' }, }, }) // Add/update a disk at runtime Storage.configure('backups', { driver: 's3', bucket: 'my-backups', region: 'eu-west-1' }) // Change default disk Storage.setDefaultDisk('s3') // Query configuration Storage.getDefaultDisk() // 'local' Storage.getConfiguredDisks() // ['local', 'public', 's3'] Storage.getDiskConfig('s3') // S3DiskConfig object // Reset all caches (useful for testing) Storage.reset() ``` ### Built-in Disk Configurations The facade auto-configures these disks from `config/filesystems.ts`: - **`local`** -- `driver: 'local'`, root: `<project>/storage/app`, visibility: from config (default `'private'`) - **`public`** -- `driver: 'local'`, root: `<project>/public`, url: the application origin, visibility: `'public'` - **`s3`** -- only added if `s3.bucket` is configured in filesystems config ## StorageAdapter Interface All adapters implement this interface: ```typescript interface StorageAdapter { write(path: string, contents: FileContents): Promise<PutResult> read(path: string): Promise<FileContents> readToString(path: string): Promise<string> readToBuffer(path: string): Promise<Buffer> readToUint8Array(path: string): Promise<Uint8Array> deleteFile(path: string): Promise<void> deleteDirectory(path: string): Promise<void> createDirectory(path: string): Promise<void> moveFile(from: string, to: string): Promise<void> copyFile(from: string, to: string): Promise<void> stat(path: string): Promise<StatEntry> list(path: string, options?: ListOptions): DirectoryListing // AsyncIterable changeVisibility(path: string, visibility: Visibility): Promise<void> visibility(path: string): Promise<Visibility> fileExists(path: string): Promise<boolean> directoryExists(path: string): Promise<boolean> publicUrl(path: string, options?: PublicUrlOptions): Promise<string> temporaryUrl(path: string, options: TemporaryUrlOptions): Promise<string> checksum(path: string, options?: ChecksumOptions): Promise<string> mimeType(path: string, options?: MimeTypeOptions): Promise<string> lastModified(path: string): Promise<number> fileSize(path: string): Promise<number> } ``` ### Local Adapter (`LocalStorageAdapter`) ```typescript import { createLocalStorage } from '@stacksjs/storage' const local = createLocalStorage({ root: './storage' }) ``` - Uses Node.js `fs/promises` for file operations - Path traversal protection: throws if resolved path escapes the root directory - `write()` auto-creates parent directories with `mkdir({ recursive: true })` - `moveFile()` uses `fs.rename()`; `copyFile()` uses `fs.copyFile()` - `checksum()` uses `Bun.CryptoHasher` (default algorithm: `sha256`) - `temporaryUrl()` generates HMAC-signed URLs using `APP_KEY` env var - `changeVisibility()` changes POSIX modes: public files/directories use 0644/0755, private use 0600/0700 - `visibility()` reads mode bits; a world-readable entry is public, other modes collapse to private - MIME type detection is extension-based (supports txt, html, css, js, json, xml, pdf, zip, jpg, jpeg, png, gif, svg, mp4, mp3, wav) - `list()` returns an async iterable; supports `deep: true` for recursive listing ### S3 Adapter (`S3StorageAdapter`) ```typescript import { createS3Storage, S3StorageAdapter } from '@stacksjs/storage' import { S3Client } from '@stacksjs/ts-cloud' const client = new S3Client('us-east-1') const s3 = new S3StorageAdapter(client, { bucket: 'my-bucket', region: 'us-east-1', prefix: 'uploads/' }) ``` - Uses `@stacksjs/ts-cloud` S3Client for AWS operations - Supports key prefix: all paths are prefixed with `config.prefix` - `write()` auto-detects MIME type from extension and sets `contentType` - `createDirectory()` is a no-op (S3 directories are implicit) - `moveFile()` = `copyFile()` + `deleteFile()` - `deleteDirectory()` lists all objects with prefix and deletes them in bulk - `temporaryUrl()` uses `client.getSignedUrl()` for pre-signed URLs - `checksum()` downloads the file content and hashes with `Bun.CryptoHasher` - `publicUrl()` defaults to `https://<bucket>.s3.<region>.amazonaws.com/<key>` - `list()` supports pagination via continuation tokens; `deep: true` uses `listAllObjects()` - `fileExists()` uses `headObject()` and catches 404/NoSuchKey/NotFound errors Every S3-COMPATIBLE provider is an `s3` disk with a different endpoint, and has a preset that fills it in: `r2Disk(bucket, accountId)`, `gcsDisk(bucket)`, `filebaseDisk(bucket)`, `backblazeDisk(bucket, region)`, `hetznerDisk(bucket, location)`. R2 additionally needs `url` set, because its API host never serves public objects. ### Azure Adapter (`AzureBlobStorageAdapter`) ```typescript import { azureDisk, AzureBlobStorageAdapter } from '@stacksjs/storage' const azure = new AzureBlobStorageAdapter({ account: 'mystorageaccount', accountKey: process.env.AZURE_STORAGE_ACCOUNT_KEY, container: 'uploads', prefix: 'tenant-7', }) ``` Its own driver rather than an `s3` disk with an endpoint, because Azure is the one provider with no S3-compatible API. - Speaks the blob REST API over `fetch`; no SDK, and no lazy-client dance - Shared Key authorization for server-side calls, service SAS for signed URLs, both in `azure-signing.ts` and tested there directly - `changeVisibility()` THROWS -- Azure has no per-blob ACL. Public access is a container property, so `visibility()` reports the container's level, and a per-object grant is `signedUrl()` - `putStream()` stages blocks (Put Block) and commits one Put Block List, so the write is atomic; block ids are zero-padded to a fixed width because Azure rejects unequal-length ids - `getStream()` is genuinely incremental, unlike the S3 adapter's buffered one - `createDirectory()` is a no-op and `deleteDirectory()` deletes by prefix, same as S3 - `endpoint` defaults to `https://<account>.blob.core.windows.net`; point it at `http://127.0.0.1:10000/devstoreaccount1` for Azurite - A `sasToken` disk can read and write but cannot sign URLs -- minting a SAS needs the account key, and re-serving the configured token would hand out its full grant ## File Uploads (UploadedFile) ```typescript import { UploadedFile, uploadedFile, uploadedFiles } from '@stacksjs/storage' // Create from native File object const file = uploadedFile(nativeFile) const files = uploadedFiles([file1, file2]) // Properties file.name // original filename file.extension // lowercase extension without dot