Skip to content

Occasional data loss with .write_all on io::stdout() without explicit .flush #7174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
kpcyrd opened this issue Feb 25, 2025 · 1 comment · May be fixed by #7178
Open

Occasional data loss with .write_all on io::stdout() without explicit .flush #7174

kpcyrd opened this issue Feb 25, 2025 · 1 comment · May be fixed by #7178
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-io Module: tokio/io

Comments

@kpcyrd
Copy link

kpcyrd commented Feb 25, 2025

Version

│   │   ├── tokio v1.43.0
│   │   │   └── tokio-macros v2.5.0 (proc-macro)
│   │   └── tokio-util v0.7.13
│   │       └── tokio v1.43.0 (*)
│   ├── tokio v1.43.0 (*)
│   ├── tokio-rustls v0.24.1
│   │   └── tokio v1.43.0 (*)
│   ├── tokio-stream v0.1.17
│   │   └── tokio v1.43.0 (*)
│   ├── tokio-util v0.7.13 (*)
│   │   ├── tokio v1.43.0 (*)
│   │   │   ├── tokio v1.43.0 (*)
│   │   ├── tokio v1.43.0 (*)
│   │   ├── tokio-rustls v0.26.1
│   │   │   └── tokio v1.43.0 (*)
│   ├── tokio v1.43.0 (*)
│   ├── tokio-rustls v0.26.1 (*)
│   ├── tokio-socks v0.5.2
│   │   └── tokio v1.43.0 (*)
│   ├── tokio-util v0.7.13 (*)
│   │   ├── tokio v1.43.0 (*)
├── tokio v1.43.0 (*)
├── tokio-socks v0.5.2 (*)

Platform

Linux redacted 6.7.6-hardened1-1-hardened #1 SMP PREEMPT_DYNAMIC Sat, 24 Feb 2024 23:47:28 +0000 x86_64 GNU/Linux

Description

I had code along the lines of:

let value: Cow<'_, [u8]> = /* some stuff here */;
stdout.write_all(&value).await?;

The program exits almost immediately afterwards. Upon testing I noticed it would sometimes fail to print anything, I first assumed a bug in my implementation but adding stdout.flush().await?; made the problem go away, and ##rust on irc confirmed stdout is buffered in Rust:

let value: Cow<'_, [u8]> = /* some stuff here */;
stdout.write_all(&value).await?;
stdout.flush().await?;

Usually the Rust standard library flushes stdout on exit:

12:41 <Alexendoo> kpcyrd: The unwritten output is flushed at a normal exit though (return from main/std::process::exit)

It seems this doesn't work well with tokio, possibly because it would require an .await, although #[tokio::main] should be able to do this for me, so I don't have to sprinkle stdout.flush().await?; throughout my codebase to avoid silent data loss.
@kpcyrd kpcyrd added A-tokio Area: The main tokio crate C-bug Category: This is a bug. labels Feb 25, 2025
@Darksonn Darksonn added the M-io Module: tokio/io label Feb 25, 2025
@Darksonn
Copy link
Contributor

Darksonn commented Feb 25, 2025 •
edited
Loading

Thanks for this issue. I agree that stdout and stderr should use spawn_mandatory_blocking instead of spawn_blocking, just like tokio::fs does.

Of course, we cannot fully guarantee that the operation will really happen. If you call std::process::abort or use shutdown_timeout, then the printing may not happen.

@maminrayej maminrayej linked a pull request Feb 25, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-io Module: tokio/io
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

io: mandate stdio operations maminrayej/tokio
2 participants
@Darksonn @kpcyrd