Commit Graph
5 Commits
Author SHA1 Message Date
Dmitriy Kovalenko c227654800 Consolidate Rust unit tests under dedicated test modules (#56247)
## What changed

Move inline crate tests into `lib_tests.rs` and nest related test suites under each crate's `tests` module. Update imports, subprocess test filters, logging targets, and snapshot names to match the new module paths.

Keep test helpers accessible where needed and preserve the schema writer's exact test name used by release tooling.

GitOrigin-RevId: de55eaae66dce4941abb33dd0b2f170093daa8f9
2026-10-22 19:36:47 +00:00
jif 4be9cfdf30 Improve diagnostics for compressed rollout read failures (#55545)
## Why

Undecodable compressed rollouts need diagnostic details and their original bytes preserved for investigation.

## What changed

- Log compressed file size, first-frame content size, decoded byte count, and whether the open file changed when a compressed rollout read fails. Limit warnings to once per process per minute and exclude paths and contents.
- Allow diagnostic reports to attach the raw `.jsonl.zst` archive after a decoding failure when no preferred plain rollout exists. Retain attachment size limits and append `.zst` to filename overrides when needed.

## Testing

Add coverage for truncated streams, incomplete trailing frame headers, files truncated after opening, and diagnostic uploads that preserve undecodable archives with size limits and filename overrides.

GitOrigin-RevId: 76f7ee466f9bebdab5a91ce1894f4361749edc5b
2026-10-20 11:42:48 +00:00
Dmitriy Kovalenko 9b0a676d5d Bundle rollout attachments into a gzip tar archive (#50446)
## What changed

- Combine file-backed rollouts and buffered rollout prefixes into `rollouts.tar.gz`, keeping diagnostics as separate attachments.
- Bound the archive and its upload envelope by the existing size limit, skip invalid or duplicate archive filenames, and fall back to individual rollout attachments if archiving fails.
- Add `rollout_archive_filename` to the report thread index so rollout filenames identify archive entries or fallback attachments.

## Testing

Add an upload test that extracts the archive and verifies the filenames and contents of both a compressed file-backed rollout and a buffered rollout. Update the thread index test to check the archive filename.

GitOrigin-RevId: 973da6eec2249b04ef7992b5e1e4fd27f044b80f
2026-10-02 22:47:52 +00:00
chess 242c5ce01c Preserve whole diagnostic attachments and report incomplete uploads (#44606)
## Why

Truncating diagnostic attachments can corrupt compressed archives, while skipping unreadable files can hide incomplete diagnostic delivery.

## What changed

- Remove attachment truncation and the 40,000,000-byte wire cap. Enforce the 200 MiB decoded envelope limit, including framing, and report oversized or unreadable attachments as upload failures.
- Use gzip only when it reduces envelope size, and set `Content-Encoding` according to the request bytes for both interactive and report uploads.
- Document that successful uploads mean HTTP acceptance, not durable attachment storage, and that callers must retain source files.

## Testing

Add coverage for incomplete uploads caused by missing or oversized files. Update large-file coverage to verify byte-for-byte delivery of JSONL files and a gzip archive exceeding the former wire cap, including successful archive decompression.

GitOrigin-RevId: a5316e9f407f16ee5a3dfb3f15122b1db51d6a94
2026-09-10 18:10:26 +00:00
jif 0df1daf526 Attach compressed rollouts to diagnostic reports as JSONL (#44175)
## Why

Diagnostic attachment reads assume the queued file path still exists and contains plain bytes. Compressed rollouts can therefore be omitted when only the logical `.jsonl` path is available, or attached as compressed data when a `.jsonl.zst` path is supplied.

## What changed

- Read rollout attachments through a bounded decoder that resolves plain or compressed representations without materializing a durable JSONL file.
- Use canonical `.jsonl` filenames for attachments and app-server report metadata, while preserving filename overrides.
- Apply size limits to decoded bytes and preserve JSONL prefix truncation.

## Testing

Add regression tests for compressed attachments, representation changes after queuing, plain-sibling preference, filename overrides, decoded size limits, truncation, nonregular files, and unrelated `.zst` attachments.

GitOrigin-RevId: b30f7dd08a741b0c99283460a1ce8933d2920ddf
2026-09-09 15:12:19 +00:00