Compare commits
10 Commits
b916098852
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
285a42a7a3
|
|||
|
aee4bc2eaa
|
|||
|
|
193359f0d7 | ||
|
|
bb0aae6d65 | ||
|
|
aa94e07a1f | ||
|
|
de022ca8f0 | ||
|
|
12554f2f92 | ||
|
|
e4e888a6c4 | ||
|
|
8451013b23 | ||
|
|
3e2096457f |
121
CLAUDE.md
Normal file
121
CLAUDE.md
Normal file
@@ -0,0 +1,121 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
`cull-gmail` is a Rust-based Gmail management library and CLI tool that enables automated email culling operations through the Gmail API. It supports message querying, filtering, batch operations (trash/delete), and rule-based retention policies.
|
||||||
|
|
||||||
|
## Development Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build the project
|
||||||
|
cargo build
|
||||||
|
|
||||||
|
# Run all tests
|
||||||
|
cargo test
|
||||||
|
|
||||||
|
# Run specific test file
|
||||||
|
cargo test --test gmail_client_unit_tests
|
||||||
|
|
||||||
|
# Run ignored integration tests (Gmail API integration)
|
||||||
|
cargo test --test gmail_message_list_integration -- --ignored
|
||||||
|
|
||||||
|
# Format code
|
||||||
|
cargo fmt
|
||||||
|
|
||||||
|
# Run linter
|
||||||
|
cargo clippy
|
||||||
|
|
||||||
|
# Check for security vulnerabilities
|
||||||
|
cargo audit
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Workspace Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
cull-gmail/
|
||||||
|
├── crates/cull-gmail/ # Library crate (public API)
|
||||||
|
│ ├── src/
|
||||||
|
│ │ ├── cli/ # CLI command implementations
|
||||||
|
│ │ │ ├── main.rs # CLI entry point
|
||||||
|
│ │ │ ├── labels_cli.rs # labels subcommand
|
||||||
|
│ │ │ ├── messages_cli.rs # messages subcommand
|
||||||
|
│ │ │ ├── rules_cli.rs # rules subcommand
|
||||||
|
│ │ │ ├── init_cli.rs # init subcommand
|
||||||
|
│ │ │ └── token_cli.rs # token subcommand
|
||||||
|
│ │ ├── client_config.rs # OAuth2 configuration
|
||||||
|
│ │ ├── gmail_client.rs # Gmail API client
|
||||||
|
│ │ ├── message_list.rs # Message list management
|
||||||
|
│ │ ├── rules.rs # Rules configuration
|
||||||
|
│ │ ├── retention.rs # Retention policy definitions
|
||||||
|
│ │ ├── rule_processor.rs # Rule execution logic
|
||||||
|
│ │ ├── eol_action.rs # End-of-life actions (trash/delete)
|
||||||
|
│ │ ├── error.rs # Error types
|
||||||
|
│ │ └── utils.rs # Utility functions
|
||||||
|
│ └── tests/ # Test files
|
||||||
|
├── docs/ # Documentation
|
||||||
|
└── .circleci/ # CI configuration
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Components
|
||||||
|
|
||||||
|
**GmailClient** (`gmail_client.rs`): Main client for Gmail API interactions. Handles authentication, message querying, and batch operations.
|
||||||
|
|
||||||
|
**Rules** (`rules.rs`): Configuration for automated retention policies with validation and execution.
|
||||||
|
|
||||||
|
**Retention** (`retention.rs`): Defines message age-based retention policies (e.g., "older_than:1y").
|
||||||
|
|
||||||
|
**RuleProcessor** (`rule_processor.rs`): Executes rules against message lists, applying appropriate actions.
|
||||||
|
|
||||||
|
**ClientConfig** (`client_config.rs`): Manages OAuth2 credentials and configuration file parsing.
|
||||||
|
|
||||||
|
**CLI** (`cli/`): Command-line interface with clap-based argument parsing.
|
||||||
|
|
||||||
|
## Gmail API Integration
|
||||||
|
|
||||||
|
- Uses `google-gmail1` crate (version 7.0.0) with `yup-oauth2` and `aws-lc-rs` features
|
||||||
|
- OAuth2 authentication with token caching in `~/.cull-gmail/gmail1/`
|
||||||
|
- Default page size: 200 messages (configurable via `DEFAULT_MAX_RESULTS`)
|
||||||
|
- Supports Gmail search syntax for queries
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Unit tests: `cargo test`
|
||||||
|
- Integration tests: Located in `crates/cull-gmail/tests/`
|
||||||
|
- Gmail API integration tests are ignored by default (`#[ignore]`) and require `--ignored` flag
|
||||||
|
- Mock tests available in test suite for unit testing
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
- Default config path: `~/.cull-gmail/cull-gmail.toml`
|
||||||
|
- OAuth2 credential file path: `~/.cull-gmail/client_secret.json`
|
||||||
|
- Rules file: `~/.cull-gmail/rules.toml`
|
||||||
|
- Environment variables override config settings (e.g., `APP_CREDENTIAL_FILE`)
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
The library uses a custom `Error` type with variants for:
|
||||||
|
- `NoLabelsFound`: Mailbox has no labels
|
||||||
|
- `LabelNotFoundInMailbox(String)`: Specific label not found
|
||||||
|
- `RuleNotFound(usize)`: Rule ID doesn't exist
|
||||||
|
- `GoogleGmail1(Box<google_gmail1::Error>)`: Gmail API errors
|
||||||
|
- `StdIO(std::io::Error)`: File I/O errors
|
||||||
|
- `Config(config::ConfigError)`: Configuration errors
|
||||||
|
|
||||||
|
## Logging
|
||||||
|
|
||||||
|
- Uses `log` crate with `env_logger`
|
||||||
|
- Verbosity controlled via CLI flags (`-v`, `-vv`, `-vvv`) and `RUST_LOG` environment variable
|
||||||
|
- Log levels: error, warn, info, debug, trace
|
||||||
|
|
||||||
|
## Code Style
|
||||||
|
|
||||||
|
- Rust edition 2024
|
||||||
|
- Minimum Rust version: 1.88
|
||||||
|
- Clippy lints configured in workspace
|
||||||
|
- Follows official Rust style guide
|
||||||
|
- Use `cargo fmt` for formatting
|
||||||
|
- Use `cargo clippy` for linting
|
||||||
2
Cargo.lock
generated
2
Cargo.lock
generated
@@ -457,7 +457,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "cull-gmail"
|
name = "cull-gmail"
|
||||||
version = "0.1.5"
|
version = "0.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"assert_cmd",
|
"assert_cmd",
|
||||||
"assert_fs",
|
"assert_fs",
|
||||||
|
|||||||
16
PRLOG.md
16
PRLOG.md
@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [0.1.7] - 2026-03-14
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- allow same label in trash and delete rules in validate(pr [#178])
|
||||||
|
|
||||||
|
## [0.1.6] - 2026-03-13
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- add rules validate subcommand(pr [#177])
|
||||||
|
|
||||||
## [0.1.5] - 2026-03-13
|
## [0.1.5] - 2026-03-13
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
@@ -509,6 +521,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
[#174]: https://github.com/jerus-org/cull-gmail/pull/174
|
[#174]: https://github.com/jerus-org/cull-gmail/pull/174
|
||||||
[#175]: https://github.com/jerus-org/cull-gmail/pull/175
|
[#175]: https://github.com/jerus-org/cull-gmail/pull/175
|
||||||
[#176]: https://github.com/jerus-org/cull-gmail/pull/176
|
[#176]: https://github.com/jerus-org/cull-gmail/pull/176
|
||||||
|
[#177]: https://github.com/jerus-org/cull-gmail/pull/177
|
||||||
|
[#178]: https://github.com/jerus-org/cull-gmail/pull/178
|
||||||
|
[0.1.7]: https://github.com/jerus-org/cull-gmail/compare/v0.1.6...v0.1.7
|
||||||
|
[0.1.6]: https://github.com/jerus-org/cull-gmail/compare/v0.1.5...v0.1.6
|
||||||
[0.1.5]: https://github.com/jerus-org/cull-gmail/compare/v0.1.4...v0.1.5
|
[0.1.5]: https://github.com/jerus-org/cull-gmail/compare/v0.1.4...v0.1.5
|
||||||
[0.1.4]: https://github.com/jerus-org/cull-gmail/compare/v0.1.3...v0.1.4
|
[0.1.4]: https://github.com/jerus-org/cull-gmail/compare/v0.1.3...v0.1.4
|
||||||
[0.1.3]: https://github.com/jerus-org/cull-gmail/compare/v0.1.2...v0.1.3
|
[0.1.3]: https://github.com/jerus-org/cull-gmail/compare/v0.1.2...v0.1.3
|
||||||
|
|||||||
72
README.md
72
README.md
@@ -53,8 +53,9 @@ Get started with cull-gmail in minutes using the built-in setup command:
|
|||||||
- **Flexible configuration**: Support for file-based config, environment variables, and ephemeral tokens
|
- **Flexible configuration**: Support for file-based config, environment variables, and ephemeral tokens
|
||||||
- **Safety first**: Dry-run mode by default, interactive confirmations, and timestamped backups
|
- **Safety first**: Dry-run mode by default, interactive confirmations, and timestamped backups
|
||||||
- **Label management**: List and inspect Gmail labels for rule planning
|
- **Label management**: List and inspect Gmail labels for rule planning
|
||||||
- **Message operations**: Query, filter, and perform batch operations on Gmail messages
|
- **Message operations**: Query, filter, and perform batch operations on Gmail messages
|
||||||
- **Rule-based automation**: Configure retention rules with time-based filtering and automated actions
|
- **Rule-based automation**: Configure retention rules with time-based filtering and automated actions
|
||||||
|
- **Mbox analysis**: Analyze Google Takeout exports to identify top senders (efficient streaming, no API needed)
|
||||||
- **Token portability**: Export/import OAuth2 tokens for containerized and CI/CD environments
|
- **Token portability**: Export/import OAuth2 tokens for containerized and CI/CD environments
|
||||||
|
|
||||||
### Running the optional Gmail integration test
|
### Running the optional Gmail integration test
|
||||||
@@ -201,9 +202,12 @@ cull-gmail [OPTIONS] [COMMAND]
|
|||||||
|
|
||||||
### Commands
|
### Commands
|
||||||
|
|
||||||
|
- `init`: Initialize configuration and OAuth2 credentials
|
||||||
- `labels`: List available Gmail labels
|
- `labels`: List available Gmail labels
|
||||||
- `messages`: Query and operate on messages
|
- `messages`: Query and operate on messages
|
||||||
- `rules`: Configure and run retention rules
|
- `rules`: Configure and run retention rules
|
||||||
|
- `analytics`: Analyze mbox files for sender statistics
|
||||||
|
- `token`: Export and import OAuth2 tokens
|
||||||
|
|
||||||
## Command Reference
|
## Command Reference
|
||||||
|
|
||||||
@@ -370,6 +374,70 @@ cull-gmail rules run --execute --skip-trash
|
|||||||
cull-gmail rules run --execute --skip-delete
|
cull-gmail rules run --execute --skip-delete
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Analytics Command
|
||||||
|
|
||||||
|
Analyze Google Takeout mbox files to identify top senders by message count.
|
||||||
|
|
||||||
|
**Note**: This command does NOT require Gmail API access. It efficiently streams local mbox files with minimal memory usage, making it suitable for analyzing large exports (60GB+).
|
||||||
|
|
||||||
|
#### Syntax
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cull-gmail analytics [OPTIONS] <MBOX_FILE>
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Arguments
|
||||||
|
|
||||||
|
- `<MBOX_FILE>`: Path to mbox file to analyze (typically from Google Takeout)
|
||||||
|
|
||||||
|
#### Options
|
||||||
|
|
||||||
|
- `-n, --top <TOP>`: Number of top senders to display [default: 10]
|
||||||
|
|
||||||
|
#### Examples
|
||||||
|
|
||||||
|
**Show top 10 senders from a Google Takeout mbox**:
|
||||||
|
```bash
|
||||||
|
cull-gmail analytics ~/takeout/All\ mail\ Including\ Spam\ and\ Trash.mbox
|
||||||
|
```
|
||||||
|
|
||||||
|
**Show top 20 senders**:
|
||||||
|
```bash
|
||||||
|
cull-gmail analytics -n 20 ~/takeout/All\ mail.mbox
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example Output**:
|
||||||
|
```
|
||||||
|
[INFO] Scanned 1234567 messages total.
|
||||||
|
Top 10 senders:
|
||||||
|
45678 newsletter@example.com
|
||||||
|
23456 promotions@example.com
|
||||||
|
18901 notifications@example.com
|
||||||
|
12345 support@example.com
|
||||||
|
9876 marketing@example.com
|
||||||
|
8765 updates@example.com
|
||||||
|
7654 alerts@example.com
|
||||||
|
6543 digests@example.com
|
||||||
|
5432 reports@example.com
|
||||||
|
4321 announcements@example.com
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Use Cases
|
||||||
|
|
||||||
|
- Identify top email senders in your mailbox before configuring rules
|
||||||
|
- Analyze historical email patterns from a full account export
|
||||||
|
- Find unexpected high-volume senders for further investigation
|
||||||
|
- Plan email retention policies based on actual sender frequency
|
||||||
|
|
||||||
|
#### Getting a Google Takeout mbox File
|
||||||
|
|
||||||
|
1. Visit [Google Takeout](https://takeout.google.com)
|
||||||
|
2. Select "Gmail" and choose the desired email account
|
||||||
|
3. Select export format "Standard" (generates .mbox files)
|
||||||
|
4. Download the export (can be very large - multiple parts possible)
|
||||||
|
5. Extract/combine the mbox files if needed
|
||||||
|
6. Use `cull-gmail analytics` on the mbox file
|
||||||
|
|
||||||
## Gmail Query Syntax
|
## Gmail Query Syntax
|
||||||
|
|
||||||
The `-Q, --query` option supports Gmail's powerful search syntax:
|
The `-Q, --query` option supports Gmail's powerful search syntax:
|
||||||
@@ -664,7 +732,7 @@ Add the library to your `Cargo.toml`:
|
|||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
cull-gmail = "0.1.5"
|
cull-gmail = "0.1.7"
|
||||||
tokio = { version = "1.0", features = ["macros", "rt-multi-thread"] }
|
tokio = { version = "1.0", features = ["macros", "rt-multi-thread"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "cull-gmail"
|
name = "cull-gmail"
|
||||||
description = "Cull emails from a gmail account using the gmail API"
|
description = "Cull emails from a gmail account using the gmail API"
|
||||||
version = "0.1.5"
|
version = "0.1.7"
|
||||||
edition.workspace = true
|
edition.workspace = true
|
||||||
authors.workspace = true
|
authors.workspace = true
|
||||||
rust-version.workspace = true
|
rust-version.workspace = true
|
||||||
|
|||||||
@@ -664,7 +664,7 @@ Add the library to your `Cargo.toml`:
|
|||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
cull-gmail = "0.1.5"
|
cull-gmail = "0.1.7"
|
||||||
tokio = { version = "1.0", features = ["macros", "rt-multi-thread"] }
|
tokio = { version = "1.0", features = ["macros", "rt-multi-thread"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
89
crates/cull-gmail/src/cli/analytics_cli.rs
Normal file
89
crates/cull-gmail/src/cli/analytics_cli.rs
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
//! # Analytics CLI Module
|
||||||
|
//!
|
||||||
|
//! Analyze mbox files to extract sender statistics.
|
||||||
|
//! Efficiently processes large Google Takeout exports without loading files into memory.
|
||||||
|
|
||||||
|
use clap::Parser;
|
||||||
|
use std::{collections::HashMap, fs::File, io::{BufRead, BufReader}, path::PathBuf};
|
||||||
|
use cull_gmail::Result;
|
||||||
|
|
||||||
|
/// Analyze an mbox file for sender statistics.
|
||||||
|
///
|
||||||
|
/// Parses Google Takeout mbox files to count messages by sender.
|
||||||
|
/// Efficient memory usage: uses streaming line-by-line parsing even for 60GB+ files.
|
||||||
|
///
|
||||||
|
/// # Examples
|
||||||
|
///
|
||||||
|
/// ```bash
|
||||||
|
/// # Show top 10 senders (default)
|
||||||
|
/// cull-gmail analytics ~/takeout/All\ mail\ Including\ Spam\ and\ Trash.mbox
|
||||||
|
///
|
||||||
|
/// # Show top 20 senders
|
||||||
|
/// cull-gmail analytics -n 20 ~/takeout/All\ mail.mbox
|
||||||
|
/// ```
|
||||||
|
#[derive(Debug, Parser)]
|
||||||
|
pub struct AnalyticsCli {
|
||||||
|
/// Path to mbox file to analyze.
|
||||||
|
#[arg(value_name = "MBOX_FILE", help = "Path to mbox file to analyze")]
|
||||||
|
pub mbox_file: PathBuf,
|
||||||
|
|
||||||
|
/// Number of top senders to display.
|
||||||
|
#[arg(
|
||||||
|
short = 'n',
|
||||||
|
long = "top",
|
||||||
|
default_value = "10",
|
||||||
|
help = "Number of top senders to display"
|
||||||
|
)]
|
||||||
|
pub top: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl AnalyticsCli {
|
||||||
|
pub fn run(&self) -> Result<()> {
|
||||||
|
let file = File::open(&self.mbox_file)?;
|
||||||
|
// Use 1MB buffer for efficient sequential reads on large files (e.g. 60GB takeout)
|
||||||
|
let reader = BufReader::with_capacity(1024 * 1024, file);
|
||||||
|
let mut counts: HashMap<String, usize> = HashMap::new();
|
||||||
|
let mut current_from: Option<String> = None;
|
||||||
|
let mut in_headers = false;
|
||||||
|
let mut message_count: usize = 0;
|
||||||
|
|
||||||
|
for line in reader.lines() {
|
||||||
|
let line = line?;
|
||||||
|
if line.starts_with("From ") {
|
||||||
|
if let Some(from) = current_from.take() {
|
||||||
|
*counts.entry(from).or_insert(0) += 1;
|
||||||
|
}
|
||||||
|
in_headers = true;
|
||||||
|
message_count += 1;
|
||||||
|
if message_count % 10_000 == 0 {
|
||||||
|
eprint!("\r[INFO] Scanned {} messages...", message_count);
|
||||||
|
}
|
||||||
|
} else if line.is_empty() {
|
||||||
|
in_headers = false;
|
||||||
|
} else if in_headers && line.to_lowercase().starts_with("from:") {
|
||||||
|
current_from = Some(extract_email(line[5..].trim()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(from) = current_from {
|
||||||
|
*counts.entry(from).or_insert(0) += 1;
|
||||||
|
}
|
||||||
|
eprintln!("\r[INFO] Scanned {} messages total.", message_count);
|
||||||
|
|
||||||
|
let mut sorted: Vec<_> = counts.iter().collect();
|
||||||
|
sorted.sort_by(|a, b| b.1.cmp(a.1));
|
||||||
|
println!("Top {} senders:", self.top.min(sorted.len()));
|
||||||
|
for (sender, count) in sorted.iter().take(self.top) {
|
||||||
|
println!(" {:6} {}", count, sender);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn extract_email(from: &str) -> String {
|
||||||
|
if let Some(start) = from.rfind('<') {
|
||||||
|
if let Some(end) = from[start..].find('>') {
|
||||||
|
return from[start + 1..start + end].to_lowercase();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
from.to_lowercase()
|
||||||
|
}
|
||||||
@@ -1005,8 +1005,8 @@ impl InitCli {
|
|||||||
let config_path = parse_config_root(config_root);
|
let config_path = parse_config_root(config_root);
|
||||||
|
|
||||||
let client_config = ClientConfig::builder()
|
let client_config = ClientConfig::builder()
|
||||||
.with_credential_file(InitDefaults::credential_filename())
|
|
||||||
.with_config_path(config_path.to_string_lossy().as_ref())
|
.with_config_path(config_path.to_string_lossy().as_ref())
|
||||||
|
.with_credential_file(InitDefaults::credential_filename())
|
||||||
.build();
|
.build();
|
||||||
|
|
||||||
// Initialize Gmail client which will trigger OAuth flow if needed
|
// Initialize Gmail client which will trigger OAuth flow if needed
|
||||||
|
|||||||
@@ -111,6 +111,7 @@
|
|||||||
|
|
||||||
use clap::{Parser, Subcommand};
|
use clap::{Parser, Subcommand};
|
||||||
|
|
||||||
|
mod analytics_cli;
|
||||||
mod init_cli;
|
mod init_cli;
|
||||||
mod labels_cli;
|
mod labels_cli;
|
||||||
mod messages_cli;
|
mod messages_cli;
|
||||||
@@ -121,6 +122,7 @@ use config::Config;
|
|||||||
use cull_gmail::{ClientConfig, EolAction, GmailClient, MessageList, Result, RuleProcessor, Rules};
|
use cull_gmail::{ClientConfig, EolAction, GmailClient, MessageList, Result, RuleProcessor, Rules};
|
||||||
use std::{env, error::Error as stdError};
|
use std::{env, error::Error as stdError};
|
||||||
|
|
||||||
|
use analytics_cli::AnalyticsCli;
|
||||||
use init_cli::InitCli;
|
use init_cli::InitCli;
|
||||||
use labels_cli::LabelsCli;
|
use labels_cli::LabelsCli;
|
||||||
use messages_cli::MessagesCli;
|
use messages_cli::MessagesCli;
|
||||||
@@ -213,6 +215,13 @@ enum SubCmds {
|
|||||||
/// environment variables for container deployments and CI/CD pipelines.
|
/// environment variables for container deployments and CI/CD pipelines.
|
||||||
#[clap(name = "token", display_order = 4)]
|
#[clap(name = "token", display_order = 4)]
|
||||||
Token(TokenCli),
|
Token(TokenCli),
|
||||||
|
|
||||||
|
/// Analyze mbox files for sender statistics.
|
||||||
|
///
|
||||||
|
/// Parse Google Takeout mbox exports to identify top senders by message count.
|
||||||
|
/// Efficient streaming for large files (60GB+) with minimal memory usage.
|
||||||
|
#[clap(name = "analytics", display_order = 5)]
|
||||||
|
Analytics(AnalyticsCli),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// CLI application entry point with comprehensive error handling and logging setup.
|
/// CLI application entry point with comprehensive error handling and logging setup.
|
||||||
@@ -298,6 +307,11 @@ async fn run(args: Cli) -> Result<()> {
|
|||||||
return init_cli.run().await;
|
return init_cli.run().await;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Handle analytics command before loading config: it only reads local mbox files
|
||||||
|
if let Some(SubCmds::Analytics(analytics_cli)) = args.sub_command {
|
||||||
|
return analytics_cli.run();
|
||||||
|
}
|
||||||
|
|
||||||
// Handle `rules validate` before loading config: it needs no Gmail credentials.
|
// Handle `rules validate` before loading config: it needs no Gmail credentials.
|
||||||
if let Some(SubCmds::Rules(ref rules_cli)) = args.sub_command
|
if let Some(SubCmds::Rules(ref rules_cli)) = args.sub_command
|
||||||
&& let Some(result) = rules_cli.run_if_validate()
|
&& let Some(result) = rules_cli.run_if_validate()
|
||||||
@@ -327,6 +341,10 @@ async fn run(args: Cli) -> Result<()> {
|
|||||||
// This should never be reached due to early return above
|
// This should never be reached due to early return above
|
||||||
unreachable!("Init command should have been handled earlier");
|
unreachable!("Init command should have been handled earlier");
|
||||||
}
|
}
|
||||||
|
SubCmds::Analytics(_) => {
|
||||||
|
// This should never be reached due to early return above
|
||||||
|
unreachable!("Analytics command should have been handled earlier");
|
||||||
|
}
|
||||||
SubCmds::Message(messages_cli) => messages_cli.run(&mut client).await,
|
SubCmds::Message(messages_cli) => messages_cli.run(&mut client).await,
|
||||||
SubCmds::Labels(labels_cli) => labels_cli.run(client).await,
|
SubCmds::Labels(labels_cli) => labels_cli.run(client).await,
|
||||||
SubCmds::Rules(rules_cli) => {
|
SubCmds::Rules(rules_cli) => {
|
||||||
|
|||||||
@@ -289,17 +289,13 @@ impl MessagesCli {
|
|||||||
|
|
||||||
match self.action {
|
match self.action {
|
||||||
MessageAction::List => {
|
MessageAction::List => {
|
||||||
if log::max_level() >= log::Level::Info {
|
let messages = client.messages();
|
||||||
client.log_messages("", "").await
|
println!("Found {} messages matching query", messages.len());
|
||||||
} else {
|
Ok(())
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
MessageAction::Trash => client.batch_trash().await,
|
MessageAction::Trash => client.batch_trash().await,
|
||||||
MessageAction::Delete => client.batch_delete().await,
|
MessageAction::Delete => client.batch_delete().await,
|
||||||
}
|
}
|
||||||
|
|
||||||
// Ok(())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Configures the Gmail client with filtering and pagination parameters.
|
/// Configures the Gmail client with filtering and pagination parameters.
|
||||||
|
|||||||
@@ -333,7 +333,7 @@ impl ClientConfig {
|
|||||||
console.installed.unwrap()
|
console.installed.unwrap()
|
||||||
};
|
};
|
||||||
|
|
||||||
let persist_path = format!("{}/gmail1", config_root.full_path().display());
|
let persist_path = format!("{}/gmail1/tokencache.json", config_root.full_path().display());
|
||||||
|
|
||||||
Ok(ClientConfig {
|
Ok(ClientConfig {
|
||||||
config_root,
|
config_root,
|
||||||
@@ -605,7 +605,7 @@ impl ConfigBuilder {
|
|||||||
}
|
}
|
||||||
|
|
||||||
pub fn build(&self) -> ClientConfig {
|
pub fn build(&self) -> ClientConfig {
|
||||||
let persist_path = format!("{}/gmail1", self.full_path());
|
let persist_path = format!("{}/gmail1/tokencache.json", self.full_path());
|
||||||
|
|
||||||
ClientConfig {
|
ClientConfig {
|
||||||
secret: self.secret.clone(),
|
secret: self.secret.clone(),
|
||||||
@@ -889,7 +889,7 @@ mod tests {
|
|||||||
.with_config_path("/custom/path")
|
.with_config_path("/custom/path")
|
||||||
.build();
|
.build();
|
||||||
|
|
||||||
assert_eq!(config.persist_path(), "/custom/path/gmail1");
|
assert_eq!(config.persist_path(), "/custom/path/gmail1/tokencache.json");
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -906,7 +906,7 @@ mod tests {
|
|||||||
assert_eq!(secret.client_secret, "accessor-test-secret");
|
assert_eq!(secret.client_secret, "accessor-test-secret");
|
||||||
|
|
||||||
// Test persist_path() accessor
|
// Test persist_path() accessor
|
||||||
assert_eq!(config.persist_path(), "/test/path/gmail1");
|
assert_eq!(config.persist_path(), "/test/path/gmail1/tokencache.json");
|
||||||
|
|
||||||
// Test full_path() accessor
|
// Test full_path() accessor
|
||||||
assert_eq!(config.full_path(), "/test/path");
|
assert_eq!(config.full_path(), "/test/path");
|
||||||
|
|||||||
@@ -405,7 +405,9 @@ impl GmailService for GmailClient {
|
|||||||
if let Some(token) = page_token {
|
if let Some(token) = page_token {
|
||||||
call = call.page_token(&token);
|
call = call.page_token(&token);
|
||||||
}
|
}
|
||||||
|
eprintln!("[DEBUG] Making Gmail API call: messages_list(query='{}', max_results={})", query, max_results);
|
||||||
let (_response, list) = call.doit().await.map_err(Box::new)?;
|
let (_response, list) = call.doit().await.map_err(Box::new)?;
|
||||||
|
eprintln!("[DEBUG] API call completed");
|
||||||
Ok(list)
|
Ok(list)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -483,7 +485,12 @@ impl MessageList for GmailClient {
|
|||||||
|
|
||||||
/// Run the Gmail api as configured
|
/// Run the Gmail api as configured
|
||||||
async fn get_messages(&mut self, pages: u32) -> Result<()> {
|
async fn get_messages(&mut self, pages: u32) -> Result<()> {
|
||||||
|
eprintln!("[DEBUG] Starting message fetch: pages={}, query='{}', labels={:?}",
|
||||||
|
pages, self.query, self.label_ids);
|
||||||
let list = self.list_messages(None).await?;
|
let list = self.list_messages(None).await?;
|
||||||
|
eprintln!("[DEBUG] Got response: {} messages (estimated: {})",
|
||||||
|
list.messages.as_ref().map(|m| m.len()).unwrap_or(0),
|
||||||
|
list.result_size_estimate.unwrap_or(0));
|
||||||
match pages {
|
match pages {
|
||||||
1 => {}
|
1 => {}
|
||||||
0 => {
|
0 => {
|
||||||
@@ -553,6 +560,7 @@ impl MessageList for GmailClient {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn log_messages(&mut self, pre: &str, post: &str) -> Result<()> {
|
async fn log_messages(&mut self, pre: &str, post: &str) -> Result<()> {
|
||||||
|
eprintln!("[INFO] Processing {} messages for output...", self.messages.len());
|
||||||
for i in 0..self.messages.len() {
|
for i in 0..self.messages.len() {
|
||||||
let id = self.messages[i].id().to_string();
|
let id = self.messages[i].id().to_string();
|
||||||
log::trace!("{id}");
|
log::trace!("{id}");
|
||||||
@@ -575,7 +583,9 @@ impl MessageList for GmailClient {
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
log::info!("{pre}{}{post}", message.list_date_and_subject());
|
let output = format!("{pre}{}{post}", message.list_date_and_subject());
|
||||||
|
println!("{}", output);
|
||||||
|
log::info!("{}", output);
|
||||||
}
|
}
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
|
|||||||
@@ -481,12 +481,13 @@ impl RuleProcessor for GmailClient {
|
|||||||
return Ok(());
|
return Ok(());
|
||||||
}
|
}
|
||||||
|
|
||||||
self.log_messages("Message with subject `", "` permanently deleted")
|
eprintln!("[INFO] Deleting {} messages using batch delete API...", message_ids.len());
|
||||||
.await?;
|
log::info!("Permanently deleting {} messages using batch API", message_ids.len());
|
||||||
|
|
||||||
self.process_in_chunks(message_ids, EolAction::Delete)
|
self.process_in_chunks(message_ids, EolAction::Delete)
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
|
eprintln!("[INFO] Batch delete completed");
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
/// Moves all prepared messages to Gmail's trash folder using batch modify API.
|
/// Moves all prepared messages to Gmail's trash folder using batch modify API.
|
||||||
@@ -512,12 +513,13 @@ impl RuleProcessor for GmailClient {
|
|||||||
return Ok(());
|
return Ok(());
|
||||||
}
|
}
|
||||||
|
|
||||||
self.log_messages("Message with subject `", "` moved to trash")
|
eprintln!("[INFO] Moving {} messages to trash using batch modify API...", message_ids.len());
|
||||||
.await?;
|
log::info!("Moving {} messages to trash using batch API", message_ids.len());
|
||||||
|
|
||||||
self.process_in_chunks(message_ids, EolAction::Trash)
|
self.process_in_chunks(message_ids, EolAction::Trash)
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
|
eprintln!("[INFO] Batch trash completed");
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -550,6 +552,7 @@ impl RuleProcessor for GmailClient {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn call_batch_delete(&self, ids: &[String]) -> Result<()> {
|
async fn call_batch_delete(&self, ids: &[String]) -> Result<()> {
|
||||||
|
let count = ids.len();
|
||||||
let ids = Some(Vec::from(ids));
|
let ids = Some(Vec::from(ids));
|
||||||
let batch_request = BatchDeleteMessagesRequest { ids };
|
let batch_request = BatchDeleteMessagesRequest { ids };
|
||||||
log::trace!("{batch_request:#?}");
|
log::trace!("{batch_request:#?}");
|
||||||
@@ -566,11 +569,13 @@ impl RuleProcessor for GmailClient {
|
|||||||
log::trace!("Batch delete response {res:?}");
|
log::trace!("Batch delete response {res:?}");
|
||||||
|
|
||||||
res?;
|
res?;
|
||||||
|
eprintln!("[DEBUG] Batch delete completed: {} messages", count);
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn call_batch_trash(&self, ids: &[String]) -> Result<()> {
|
async fn call_batch_trash(&self, ids: &[String]) -> Result<()> {
|
||||||
|
let count = ids.len();
|
||||||
let ids = Some(Vec::from(ids));
|
let ids = Some(Vec::from(ids));
|
||||||
let add_label_ids = Some(vec![TRASH_LABEL.to_string()]);
|
let add_label_ids = Some(vec![TRASH_LABEL.to_string()]);
|
||||||
let remove_label_ids = Some(vec![INBOX_LABEL.to_string()]);
|
let remove_label_ids = Some(vec![INBOX_LABEL.to_string()]);
|
||||||
@@ -592,6 +597,7 @@ impl RuleProcessor for GmailClient {
|
|||||||
.await
|
.await
|
||||||
.map_err(Box::new)?;
|
.map_err(Box::new)?;
|
||||||
|
|
||||||
|
eprintln!("[DEBUG] Batch trash completed: {} messages", count);
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -688,7 +688,9 @@ impl Rules {
|
|||||||
/// ```
|
/// ```
|
||||||
pub fn validate(&self) -> Vec<ValidationIssue> {
|
pub fn validate(&self) -> Vec<ValidationIssue> {
|
||||||
let mut issues = Vec::new();
|
let mut issues = Vec::new();
|
||||||
let mut seen_labels: BTreeMap<String, usize> = BTreeMap::new();
|
// Key: (label, action_str) — the same label in a Trash and a Delete rule is
|
||||||
|
// intentional two-stage processing and must not be flagged as a duplicate.
|
||||||
|
let mut seen_label_actions: BTreeMap<(String, String), usize> = BTreeMap::new();
|
||||||
|
|
||||||
for rule in self.rules.values() {
|
for rule in self.rules.values() {
|
||||||
let id = rule.id();
|
let id = rule.id();
|
||||||
@@ -712,14 +714,15 @@ impl Rules {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for label in rule.labels() {
|
for label in rule.labels() {
|
||||||
if let Some(&other_id) = seen_labels.get(&label) {
|
let key = (label.clone(), rule.action_str().to_lowercase());
|
||||||
|
if let Some(&other_id) = seen_label_actions.get(&key) {
|
||||||
if other_id != id {
|
if other_id != id {
|
||||||
issues.push(ValidationIssue::DuplicateLabel {
|
issues.push(ValidationIssue::DuplicateLabel {
|
||||||
label: label.clone(),
|
label: label.clone(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
seen_labels.insert(label, id);
|
seen_label_actions.insert(key, id);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1224,6 +1227,33 @@ action = "Trash"
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_validate_same_label_different_actions_not_duplicate() {
|
||||||
|
setup_test_environment();
|
||||||
|
// A label in a Trash rule AND a Delete rule is intentional two-stage processing.
|
||||||
|
let toml_str = r#"
|
||||||
|
[rules."1"]
|
||||||
|
id = 1
|
||||||
|
retention = "w:1"
|
||||||
|
labels = ["Development/Notifications"]
|
||||||
|
action = "Trash"
|
||||||
|
|
||||||
|
[rules."2"]
|
||||||
|
id = 2
|
||||||
|
retention = "w:2"
|
||||||
|
labels = ["Development/Notifications"]
|
||||||
|
action = "Delete"
|
||||||
|
"#;
|
||||||
|
let rules: Rules = toml::from_str(toml_str).unwrap();
|
||||||
|
let issues = rules.validate();
|
||||||
|
assert!(
|
||||||
|
!issues
|
||||||
|
.iter()
|
||||||
|
.any(|i| matches!(i, ValidationIssue::DuplicateLabel { .. })),
|
||||||
|
"Same label with different actions should NOT be flagged as duplicate, got: {issues:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_validate_multiple_issues_collected() {
|
fn test_validate_multiple_issues_collected() {
|
||||||
setup_test_environment();
|
setup_test_environment();
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ Add the library to your `Cargo.toml`:
|
|||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
cull-gmail = "0.1.5"
|
cull-gmail = "0.1.7"
|
||||||
tokio = { version = "1.0", features = ["macros", "rt-multi-thread"] }
|
tokio = { version = "1.0", features = ["macros", "rt-multi-thread"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user