dstack-cloud: deploy large / externally-built (verbatim) app-compose.json#717
Merged
kvinwang merged 2 commits intoJun 7, 2026
Merged
Conversation
The shared disk image was a fixed 8M FAT32. A large app-compose.json (e.g. a self-contained app that embeds its payload inline, up to the 50M guest cap) overflows it, so mcopy fails and deploy aborts. Size the image from the actual shared-file sizes instead — floor 8M (small composes unchanged), +4M FAT metadata margin, rounded up to a whole MiB.
Add an optional App.app_compose_file: when set, its exact bytes become shared/app-compose.json instead of generating from docker_compose_file. The measured compose-hash is sha256 of the bytes on the shared disk, so an externally-built compose (e.g. a self-contained app whose hash is computed over its own serialization) must be shipped verbatim — re-serializing would diverge the hash. Generation is refactored into _emit_app_compose() so prepare and deploy stay consistent. Empty (default) is fully backward-compatible.
Contributor
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related changes to
scripts/bin/dstack-cloudthat let GCP deployments use a large or externally-builtapp-compose.json— e.g. self-contained apps that embed their payload inline (tools/sca), whose compose-hash is computed over their own serialization.1. Size the shared FAT disk to fit
app-compose.json(bug fix)The shared disk image was a fixed
8MFAT32. A largeapp-compose.json(the guest already allows up to 50M) overflows it, somcopyfails and deploy aborts with a non-obvious error. Now the image is sized from the actual shared-file sizes: floor 8M (small composes unchanged), +4M FAT metadata margin, rounded up to a whole MiB.2. Support deploying a verbatim
app-compose.json(feature)Add an optional
app_compose_fileto the app config. When set, its exact bytes becomeshared/app-compose.jsoninstead of being generated fromdocker_compose_file. The measured compose-hash issha256of the bytes on the shared disk, so an externally-built compose (e.g. fromtools/sca) must be shipped verbatim — re-serializing viajson.dumpwould diverge the hash. Generation is refactored into_emit_app_compose()soprepareanddeploystay consistent.Compatibility
Both are backward-compatible:
app_compose_filedefaults to empty → existing docker-compose deployments take the exact same path as before.dstack-cloudnever readsapp-compose.json's fields back, so a verbatim (non-docker-compose) compose doesn't affect any downstream step.Testing
shared/app-compose.jsonis byte-identical to the source (sha256 match), so compose-hash == the external builder's.runner: docker-composeas before.app_compose_file→FileNotFoundError(fail-closed).