Istio is a leading open-source service mesh that works with Kubernetes. It enables applications running in a Kubernetes cluster to deliver more business value. A service mesh takes care of securing service-to-service communication, identity-based authentication and authorization, and fine-grained traffic control. The increased flexibility that Istio provides also introduces many new metrics, logs, and traces that need to be captured and consolidated.

Istio's architecture has a central control plane that handles the overall configuration, such as routing and access policies enforced at the pod level on each inbound and outbound request. Their About Istio page has a good description and some helpful diagrams.

Istio Logging at Its Core

At its core, the Istio architecture has a single control plane that tracks all aspects of the configuration of the services within its service mesh. Using standard cloud-native logging practices (namely, outputting data to stdout and stderr), the core of Istio logs data related to its activities. For example, it can act as the certificate authority to enable mTLS between services or authorize the virtual services in the service mesh and any other related policies that need enforcement. The pod level is where enforcement occurs alongside network routing and most telemetry data. Still, the core is critical for ensuring consistency within the Kubernetes cluster and coordinating discovery data in multi-cluster deployments.

In addition to the control plane, all traffic flows in and out of an Istio service mesh needs to pass through ingress and egress gateways. These are separate containers that use httpbin out of the box. They also use the most common cloud-native approach to Kubernetes logging, which means that logs can be easily exported and shipped to a central logging platform.

Endpoint Logging 

Istio injects Envoy as a sidecar into each pod to proxy network traffic. Envoy is a highly configurable cloud-native proxy that has become the standard way for service meshes to manage networking and observability at the pod level.

One of Envoy's core observability capabilities is that it can write detailed logs of what it has been processing. By default, these logs contain the kind of details that you would expect to find in an access log, like the source address, the protocol used, and the path requested. In addition, these logs include extra details that enable cross-service request tracing, including a unique request ID and the upstream service that originated the call. By default, these logs go to stdout and stderr, allowing standard Kubernetes log collection functionality to pick them up.

You can adjust the log format if you want, and you can also configure Envoy to write to a specific file. However, that complicates the log collection and shipping process, so you only want to do it in particular circumstances.

Conclusion    

As a centralized collection point for logs, Mezmo enables consistent management and observability throughout the lifecycle of a Kubernetes cluster running Istio. Without it, you'd lose the logs created by hundreds or thousands of Envoy proxy instances as part of the cluster's application deployment cycle and all of the potential insights contained within the logs.

‍