divekit fetch

The divekit fetch command downloads repositories for a distribution into the local cache for offline access or further processing. It supports concurrent downloads, selective file fetching, and cache management.

Usage

divekit fetch [distribution-name] [flags]

Flags

• --base-url <url>: Base URL for the GitLab instance
• --token <value>: GitLab access token (required for GitLab provider)
• --distribution <name>, -d <name>: Specify which distribution to fetch (or as argument)
• --output-dir <path>: Custom output directory (default: use distribution cache)
• --files <patterns>: Specific files to download (glob patterns)
• --workers <int>: Number of concurrent downloads (default: 5)
• --dry-run: Show what would be downloaded without actually downloading
• --fresh: Clean cache before downloading
• --help: Show help information

Credentials

• Tokens are stored in your operating system’s credential manager (macOS Keychain, Windows Credential Manager, Linux Secret Service).
• Configure hosts via:

  divekit config set @hosts git-nrw https://gitlab.git.nrw/  # prompts for token and stores it in OS keychain
  

• The flags --token and --base-url are optional overrides. If omitted, Divekit resolves credentials and base URL from your configured hosts (hosts.json contains only host metadata and a token-key reference).

Description

The divekit fetch command downloads repository content for a specified distribution into the local cache. This enables offline access to repository files and can be used for batch processing or analysis.

Key features:

• Concurrent downloads with configurable worker pools
• Selective file fetching using glob patterns
• Cache management with fresh download option
• Dry-run mode for previewing what will be downloaded
• Integration with existing distribution configurations

The flow:

1. Validates the distribution configuration
2. Resolves credentials for the target GitLab instance
3. Determines the output directory (cache or custom)
4. Optionally cleans existing cache if --fresh is specified
5. Downloads repository content concurrently
6. Reports download progress and results

Examples

Basic fetch for a distribution

$ divekit fetch --distribution my-assignment

Preparing repository fetch output at /path/to/project/.divekit/cache/distributions/my-assignment/202501221504/repos
fetch: cache directory prepared (no downloads in this minimal version)

Fetch with custom output directory

$ divekit fetch --distribution my-assignment --output-dir ./custom-cache

Preparing repository fetch output at ./custom-cache
fetch: cache directory prepared (no downloads in this minimal version)

Dry run to preview downloads

$ divekit fetch --distribution my-assignment --dry-run

Preparing repository fetch output at /path/to/project/.divekit/cache/distributions/my-assignment/202501221504/repos
fetch: dry-run mode — skipping directory creation and downloads

Fetch specific files only

$ divekit fetch --distribution my-assignment --files "*.java" --files "README.md"

Preparing repository fetch output at /path/to/project/.divekit/cache/distributions/my-assignment/202501221504/repos
fetch: cache directory prepared (no downloads in this minimal version)

Fresh fetch (clean cache first)

$ divekit fetch --distribution my-assignment --fresh

Preparing repository fetch output at /path/to/project/.divekit/cache/distributions/my-assignment/202501221504/repos
fetch: failed to clean cache: remove /path/to/project/.divekit/cache/distributions/my-assignment/repos: no such file or directory
fetch: cache directory prepared (no downloads in this minimal version)

What Gets Downloaded

• Repository Content: All files from the configured repositories in the distribution
• Cache Structure: Organized by distribution name and timestamp for version tracking
• Selective Downloads: Only specified file patterns if --files is used
• Concurrent Processing: Multiple repositories downloaded simultaneously based on --workers setting

Notes

• The fetch command is currently in development and includes placeholder logic for actual downloads
• Cache location follows the pattern: .divekit/cache/distributions/<name>/<timestamp>/repos
• Use --dry-run to preview what would be downloaded before executing
• Downloaded content can be used for offline processing or analysis