I was working on a task which involved using grep for extracting specific patterns from the logs of a Docker container using the following command:

docker logs relayer | grep "Hermes"

I expected to see only the entries containing Hermes, but instead, the command output all the logs. Something was wrong. I manually searched (using Ctrl + F in the terminal) and confirmed that Hermes did appear in the logs. Also trying different flags with grep did not work. After that, I redirected the output of docker logs to a file:

docker logs relayer > logs.txt

But the file did not contain all the log entries as expected. Here is an excerpt from the file:

SUCCESS Restored key 'whale-emerynet' (agoric10emrzln03exuc9uv98mjwmp95735mjm6k2n9xm) on chain agoric-emerynet-8
SUCCESS Restored key 'whale-devnet' (agoric1khw65emzav9t0cdhj3aw9x2v7m60jekjdf4whl) on chain agoricdev-23
ERROR failed to spawn chain runtime: relayer error: RPC error to endpoint https://emerynet.rpc.agoric.net/: HTTP error: error sending request for url (https://emerynet.rpc.agoric.net/): error trying to connect: Connection reset by peer (os error 104)

Interestingly, after the first error message, no further log entries were appended to the file. Some solutions I’ve tried to resolve the problem were as follows:

Redirecting both stdout and stderr#

I learned that when we use the redirect operator >, it only redirects the standard output(stdout). So in order to ensure both stdout and stderr can captured, I needed to use:

docker logs relayer > logs.txt 2>&1

This command redirects stderr (denoted by 2) to stdout (denoted by 1), and then stdout to logs.txt. As a result, all output including errors will be captured in logs.txt.

The same can be done when using grep:

docker logs relayer 2>&1 | grep "Hermes"

Some good reads: