[chore] Simplify the OTEL setup (#4110)

# Description

This simplifies our OTEL setup by:

* Getting rid of some deprecated things.
* Using `autoexport` and letting things get configured by the `OTEL_` environment variables.
* Removing all the unnecessary config options.

## Checklist

Please put an x inside each checkbox to indicate that you've read and followed it: `[ ]` -> `[x]`

If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want).

- [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md).
- [x] I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat.
- [x] I/we have not leveraged AI to create the proposed changes.
- [x] I/we have performed a self-review of added code.
- [x] I/we have written code that is legible and maintainable by others.
- [ ] I/we have commented the added code, particularly in hard-to-understand areas.
- [x] I/we have made any necessary changes to documentation.
- [ ] I/we have added tests that cover new code.
- [x] I/we have run tests and they pass locally with the changes.
- [x] I/we have run `go fmt ./...` and `golangci-lint run`.

Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4110
Reviewed-by: tobi <kipvandenbos@noreply.codeberg.org>
Co-authored-by: Daenney <daenney@noreply.codeberg.org>
Co-committed-by: Daenney <daenney@noreply.codeberg.org>
This commit is contained in:
Daenney 2025-05-05 16:22:45 +00:00 committed by tobi
commit ecbdc4227b
145 changed files with 21740 additions and 1319 deletions

View file

@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View file

@ -0,0 +1,3 @@
# OTLP Log gRPC Exporter
[![PkgGoDev](https://pkg.go.dev/badge/go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc)](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc)

View file

@ -0,0 +1,258 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploggrpc // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc"
import (
"context"
"fmt"
"time"
"google.golang.org/genproto/googleapis/rpc/errdetails"
"google.golang.org/grpc"
"google.golang.org/grpc/backoff"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/encoding/gzip"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/retry"
collogpb "go.opentelemetry.io/proto/otlp/collector/logs/v1"
logpb "go.opentelemetry.io/proto/otlp/logs/v1"
)
// The methods of this type are not expected to be called concurrently.
type client struct {
metadata metadata.MD
exportTimeout time.Duration
requestFunc retry.RequestFunc
// ourConn keeps track of where conn was created: true if created here in
// NewClient, or false if passed with an option. This is important on
// Shutdown as conn should only be closed if we created it. Otherwise,
// it is up to the processes that passed conn to close it.
ourConn bool
conn *grpc.ClientConn
lsc collogpb.LogsServiceClient
}
// Used for testing.
var newGRPCClientFn = grpc.NewClient
// newClient creates a new gRPC log client.
func newClient(cfg config) (*client, error) {
c := &client{
exportTimeout: cfg.timeout.Value,
requestFunc: cfg.retryCfg.Value.RequestFunc(retryable),
conn: cfg.gRPCConn.Value,
}
if len(cfg.headers.Value) > 0 {
c.metadata = metadata.New(cfg.headers.Value)
}
if c.conn == nil {
// If the caller did not provide a ClientConn when the client was
// created, create one using the configuration they did provide.
dialOpts := newGRPCDialOptions(cfg)
conn, err := newGRPCClientFn(cfg.endpoint.Value, dialOpts...)
if err != nil {
return nil, err
}
// Keep track that we own the lifecycle of this conn and need to close
// it on Shutdown.
c.ourConn = true
c.conn = conn
}
c.lsc = collogpb.NewLogsServiceClient(c.conn)
return c, nil
}
func newGRPCDialOptions(cfg config) []grpc.DialOption {
userAgent := "OTel Go OTLP over gRPC logs exporter/" + Version()
dialOpts := []grpc.DialOption{grpc.WithUserAgent(userAgent)}
dialOpts = append(dialOpts, cfg.dialOptions.Value...)
// Convert other grpc configs to the dial options.
// Service config
if cfg.serviceConfig.Value != "" {
dialOpts = append(dialOpts, grpc.WithDefaultServiceConfig(cfg.serviceConfig.Value))
}
// Prioritize GRPCCredentials over Insecure (passing both is an error).
if cfg.gRPCCredentials.Value != nil {
dialOpts = append(dialOpts, grpc.WithTransportCredentials(cfg.gRPCCredentials.Value))
} else if cfg.insecure.Value {
dialOpts = append(dialOpts, grpc.WithTransportCredentials(insecure.NewCredentials()))
} else {
// Default to using the host's root CA.
dialOpts = append(dialOpts, grpc.WithTransportCredentials(
credentials.NewTLS(nil),
))
}
// Compression
if cfg.compression.Value == GzipCompression {
dialOpts = append(dialOpts, grpc.WithDefaultCallOptions(grpc.UseCompressor(gzip.Name)))
}
// Reconnection period
if cfg.reconnectionPeriod.Value != 0 {
p := grpc.ConnectParams{
Backoff: backoff.DefaultConfig,
MinConnectTimeout: cfg.reconnectionPeriod.Value,
}
dialOpts = append(dialOpts, grpc.WithConnectParams(p))
}
return dialOpts
}
// UploadLogs sends proto logs to connected endpoint.
//
// Retryable errors from the server will be handled according to any
// RetryConfig the client was created with.
//
// The otlplog.Exporter synchronizes access to client methods, and
// ensures this is not called after the Exporter is shutdown. Only thing
// to do here is send data.
func (c *client) UploadLogs(ctx context.Context, rl []*logpb.ResourceLogs) error {
select {
case <-ctx.Done():
// Do not upload if the context is already expired.
return ctx.Err()
default:
}
ctx, cancel := c.exportContext(ctx)
defer cancel()
return c.requestFunc(ctx, func(ctx context.Context) error {
resp, err := c.lsc.Export(ctx, &collogpb.ExportLogsServiceRequest{
ResourceLogs: rl,
})
if resp != nil && resp.PartialSuccess != nil {
msg := resp.PartialSuccess.GetErrorMessage()
n := resp.PartialSuccess.GetRejectedLogRecords()
if n != 0 || msg != "" {
err := fmt.Errorf("OTLP partial success: %s (%d log records rejected)", msg, n)
otel.Handle(err)
}
}
// nil is converted to OK.
if status.Code(err) == codes.OK {
// Success.
return nil
}
return err
})
}
// Shutdown shuts down the client, freeing all resources.
//
// Any active connections to a remote endpoint are closed if they were created
// by the client. Any gRPC connection passed during creation using
// WithGRPCConn will not be closed. It is the caller's responsibility to
// handle cleanup of that resource.
//
// The otlplog.Exporter synchronizes access to client methods and
// ensures this is called only once. The only thing that needs to be done
// here is to release any computational resources the client holds.
func (c *client) Shutdown(ctx context.Context) error {
c.metadata = nil
c.requestFunc = nil
c.lsc = nil
// Release the connection if we created it.
err := ctx.Err()
if c.ourConn {
closeErr := c.conn.Close()
// A context timeout error takes precedence over this error.
if err == nil && closeErr != nil {
err = closeErr
}
}
c.conn = nil
return err
}
// exportContext returns a copy of parent with an appropriate deadline and
// cancellation function based on the clients configured export timeout.
//
// It is the callers responsibility to cancel the returned context once its
// use is complete, via the parent or directly with the returned CancelFunc, to
// ensure all resources are correctly released.
func (c *client) exportContext(parent context.Context) (context.Context, context.CancelFunc) {
var (
ctx context.Context
cancel context.CancelFunc
)
if c.exportTimeout > 0 {
ctx, cancel = context.WithTimeout(parent, c.exportTimeout)
} else {
ctx, cancel = context.WithCancel(parent)
}
if c.metadata.Len() > 0 {
md := c.metadata
if outMD, ok := metadata.FromOutgoingContext(ctx); ok {
md = metadata.Join(md, outMD)
}
ctx = metadata.NewOutgoingContext(ctx, md)
}
return ctx, cancel
}
type noopClient struct{}
func newNoopClient() *noopClient {
return &noopClient{}
}
func (c *noopClient) UploadLogs(context.Context, []*logpb.ResourceLogs) error { return nil }
func (c *noopClient) Shutdown(context.Context) error { return nil }
// retryable returns if err identifies a request that can be retried and a
// duration to wait for if an explicit throttle time is included in err.
func retryable(err error) (bool, time.Duration) {
s := status.Convert(err)
return retryableGRPCStatus(s)
}
func retryableGRPCStatus(s *status.Status) (bool, time.Duration) {
switch s.Code() {
case codes.Canceled,
codes.DeadlineExceeded,
codes.Aborted,
codes.OutOfRange,
codes.Unavailable,
codes.DataLoss:
// Additionally, handle RetryInfo.
_, d := throttleDelay(s)
return true, d
case codes.ResourceExhausted:
// Retry only if the server signals that the recovery from resource exhaustion is possible.
return throttleDelay(s)
}
// Not a retry-able error.
return false, 0
}
// throttleDelay returns if the status is RetryInfo
// and the duration to wait for if an explicit throttle time is included.
func throttleDelay(s *status.Status) (bool, time.Duration) {
for _, detail := range s.Details() {
if t, ok := detail.(*errdetails.RetryInfo); ok {
return true, t.RetryDelay.AsDuration()
}
}
return false, 0
}

View file

@ -0,0 +1,653 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploggrpc // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc"
import (
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"net/url"
"os"
"strconv"
"strings"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/retry"
"go.opentelemetry.io/otel/internal/global"
)
// Default values.
var (
defaultEndpoint = "localhost:4317"
defaultTimeout = 10 * time.Second
defaultRetryCfg = retry.DefaultConfig
)
// Environment variable keys.
var (
envEndpoint = []string{
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
"OTEL_EXPORTER_OTLP_ENDPOINT",
}
envInsecure = []string{
"OTEL_EXPORTER_OTLP_LOGS_INSECURE",
"OTEL_EXPORTER_OTLP_INSECURE",
}
envHeaders = []string{
"OTEL_EXPORTER_OTLP_LOGS_HEADERS",
"OTEL_EXPORTER_OTLP_HEADERS",
}
envCompression = []string{
"OTEL_EXPORTER_OTLP_LOGS_COMPRESSION",
"OTEL_EXPORTER_OTLP_COMPRESSION",
}
envTimeout = []string{
"OTEL_EXPORTER_OTLP_LOGS_TIMEOUT",
"OTEL_EXPORTER_OTLP_TIMEOUT",
}
envTLSCert = []string{
"OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE",
"OTEL_EXPORTER_OTLP_CERTIFICATE",
}
envTLSClient = []struct {
Certificate string
Key string
}{
{
"OTEL_EXPORTER_OTLP_LOGS_CLIENT_CERTIFICATE",
"OTEL_EXPORTER_OTLP_LOGS_CLIENT_KEY",
},
{
"OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE",
"OTEL_EXPORTER_OTLP_CLIENT_KEY",
},
}
)
type fnOpt func(config) config
func (f fnOpt) applyOption(c config) config { return f(c) }
// Option applies an option to the Exporter.
type Option interface {
applyOption(config) config
}
type config struct {
endpoint setting[string]
insecure setting[bool]
tlsCfg setting[*tls.Config]
headers setting[map[string]string]
compression setting[Compression]
timeout setting[time.Duration]
retryCfg setting[retry.Config]
// gRPC configurations
gRPCCredentials setting[credentials.TransportCredentials]
serviceConfig setting[string]
reconnectionPeriod setting[time.Duration]
dialOptions setting[[]grpc.DialOption]
gRPCConn setting[*grpc.ClientConn]
}
func newConfig(options []Option) config {
var c config
for _, opt := range options {
c = opt.applyOption(c)
}
// Apply environment value and default value
c.endpoint = c.endpoint.Resolve(
getEnv[string](envEndpoint, convEndpoint),
fallback[string](defaultEndpoint),
)
c.insecure = c.insecure.Resolve(
loadInsecureFromEnvEndpoint(envEndpoint),
getEnv[bool](envInsecure, convInsecure),
)
c.tlsCfg = c.tlsCfg.Resolve(
loadEnvTLS[*tls.Config](),
)
c.headers = c.headers.Resolve(
getEnv[map[string]string](envHeaders, convHeaders),
)
c.compression = c.compression.Resolve(
getEnv[Compression](envCompression, convCompression),
)
c.timeout = c.timeout.Resolve(
getEnv[time.Duration](envTimeout, convDuration),
fallback[time.Duration](defaultTimeout),
)
c.retryCfg = c.retryCfg.Resolve(
fallback[retry.Config](defaultRetryCfg),
)
return c
}
// RetryConfig defines configuration for retrying the export of log data
// that failed.
//
// This configuration does not define any network retry strategy. That is
// entirely handled by the gRPC ClientConn.
type RetryConfig retry.Config
// WithInsecure disables client transport security for the Exporter's gRPC
// connection, just like grpc.WithInsecure()
// (https://pkg.go.dev/google.golang.org/grpc#WithInsecure) does.
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, that variable
// value will be used to determine client security. If the endpoint has a
// scheme of "http" or "unix" client security will be disabled. If both are
// set, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, client security will be used.
//
// This option has no effect if WithGRPCConn is used.
func WithInsecure() Option {
return fnOpt(func(c config) config {
c.insecure = newSetting(true)
return c
})
}
// WithEndpoint sets the target endpoint the Exporter will connect to.
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, that variable
// value will be used. If both are set, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// will take precedence.
//
// If both this option and WithEndpointURL are used, the last used option will
// take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, "localhost:4317" will be used.
//
// This option has no effect if WithGRPCConn is used.
func WithEndpoint(endpoint string) Option {
return fnOpt(func(c config) config {
c.endpoint = newSetting(endpoint)
return c
})
}
// WithEndpointURL sets the target endpoint URL the Exporter will connect to.
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, that variable
// value will be used. If both are set, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// will take precedence.
//
// If both this option and WithEndpoint are used, the last used option will
// take precedence.
//
// If an invalid URL is provided, the default value will be kept.
//
// By default, if an environment variable is not set, and this option is not
// passed, "localhost:4317" will be used.
//
// This option has no effect if WithGRPCConn is used.
func WithEndpointURL(rawURL string) Option {
u, err := url.Parse(rawURL)
if err != nil {
global.Error(err, "otlplog: parse endpoint url", "url", rawURL)
return fnOpt(func(c config) config { return c })
}
return fnOpt(func(c config) config {
c.endpoint = newSetting(u.Host)
c.insecure = insecureFromScheme(c.insecure, u.Scheme)
return c
})
}
// WithReconnectionPeriod set the minimum amount of time between connection
// attempts to the target endpoint.
//
// This option has no effect if WithGRPCConn is used.
func WithReconnectionPeriod(rp time.Duration) Option {
return fnOpt(func(c config) config {
c.reconnectionPeriod = newSetting(rp)
return c
})
}
// Compression describes the compression used for exported payloads.
type Compression int
const (
// NoCompression represents that no compression should be used.
NoCompression Compression = iota
// GzipCompression represents that gzip compression should be used.
GzipCompression
)
// WithCompressor sets the compressor the gRPC client uses.
// Supported compressor values: "gzip".
//
// If the OTEL_EXPORTER_OTLP_COMPRESSION or
// OTEL_EXPORTER_OTLP_LOGS_COMPRESSION environment variable is set, and
// this option is not passed, that variable value will be used. That value can
// be either "none" or "gzip". If both are set,
// OTEL_EXPORTER_OTLP_LOGS_COMPRESSION will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, no compression strategy will be used.
//
// This option has no effect if WithGRPCConn is used.
func WithCompressor(compressor string) Option {
return fnOpt(func(c config) config {
c.compression = newSetting(compressorToCompression(compressor))
return c
})
}
// WithHeaders will send the provided headers with each gRPC requests.
//
// If the OTEL_EXPORTER_OTLP_HEADERS or OTEL_EXPORTER_OTLP_LOGS_HEADERS
// environment variable is set, and this option is not passed, that variable
// value will be used. The value will be parsed as a list of key value pairs.
// These pairs are expected to be in the W3C Correlation-Context format
// without additional semi-colon delimited metadata (i.e. "k1=v1,k2=v2"). If
// both are set, OTEL_EXPORTER_OTLP_LOGS_HEADERS will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, no user headers will be set.
func WithHeaders(headers map[string]string) Option {
return fnOpt(func(c config) config {
c.headers = newSetting(headers)
return c
})
}
// WithTLSCredentials sets the gRPC connection to use creds.
//
// If the OTEL_EXPORTER_OTLP_CERTIFICATE or
// OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE environment variable is set, and
// this option is not passed, that variable value will be used. The value will
// be parsed the filepath of the TLS certificate chain to use. If both are
// set, OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, no TLS credentials will be used.
//
// This option has no effect if WithGRPCConn is used.
func WithTLSCredentials(credential credentials.TransportCredentials) Option {
return fnOpt(func(c config) config {
c.gRPCCredentials = newSetting(credential)
return c
})
}
// WithServiceConfig defines the default gRPC service config used.
//
// This option has no effect if WithGRPCConn is used.
func WithServiceConfig(serviceConfig string) Option {
return fnOpt(func(c config) config {
c.serviceConfig = newSetting(serviceConfig)
return c
})
}
// WithDialOption sets explicit grpc.DialOptions to use when establishing a
// gRPC connection. The options here are appended to the internal grpc.DialOptions
// used so they will take precedence over any other internal grpc.DialOptions
// they might conflict with.
// The [grpc.WithBlock], [grpc.WithTimeout], and [grpc.WithReturnConnectionError]
// grpc.DialOptions are ignored.
//
// This option has no effect if WithGRPCConn is used.
func WithDialOption(opts ...grpc.DialOption) Option {
return fnOpt(func(c config) config {
c.dialOptions = newSetting(opts)
return c
})
}
// WithGRPCConn sets conn as the gRPC ClientConn used for all communication.
//
// This option takes precedence over any other option that relates to
// establishing or persisting a gRPC connection to a target endpoint. Any
// other option of those types passed will be ignored.
//
// It is the callers responsibility to close the passed conn. The Exporter
// Shutdown method will not close this connection.
func WithGRPCConn(conn *grpc.ClientConn) Option {
return fnOpt(func(c config) config {
c.gRPCConn = newSetting(conn)
return c
})
}
// WithTimeout sets the max amount of time an Exporter will attempt an export.
//
// This takes precedence over any retry settings defined by WithRetry. Once
// this time limit has been reached the export is abandoned and the log
// data is dropped.
//
// If the OTEL_EXPORTER_OTLP_TIMEOUT or OTEL_EXPORTER_OTLP_LOGS_TIMEOUT
// environment variable is set, and this option is not passed, that variable
// value will be used. The value will be parsed as an integer representing the
// timeout in milliseconds. If both are set,
// OTEL_EXPORTER_OTLP_LOGS_TIMEOUT will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, a timeout of 10 seconds will be used.
func WithTimeout(duration time.Duration) Option {
return fnOpt(func(c config) config {
c.timeout = newSetting(duration)
return c
})
}
// WithRetry sets the retry policy for transient retryable errors that are
// returned by the target endpoint.
//
// If the target endpoint responds with not only a retryable error, but
// explicitly returns a backoff time in the response, that time will take
// precedence over these settings.
//
// These settings do not define any network retry strategy. That is entirely
// handled by the gRPC ClientConn.
//
// If unset, the default retry policy will be used. It will retry the export
// 5 seconds after receiving a retryable error and increase exponentially
// after each error for no more than a total time of 1 minute.
func WithRetry(rc RetryConfig) Option {
return fnOpt(func(c config) config {
c.retryCfg = newSetting(retry.Config(rc))
return c
})
}
// convCompression returns the parsed compression encoded in s. NoCompression
// and an errors are returned if s is unknown.
func convCompression(s string) (Compression, error) {
switch s {
case "gzip":
return GzipCompression, nil
case "none", "":
return NoCompression, nil
}
return NoCompression, fmt.Errorf("unknown compression: %s", s)
}
// convEndpoint converts s from a URL string to an endpoint if s is a valid
// URL. Otherwise, "" and an error are returned.
func convEndpoint(s string) (string, error) {
u, err := url.Parse(s)
if err != nil {
return "", err
}
return u.Host, nil
}
// convInsecure converts s from string to bool without case sensitivity.
// If s is not valid returns error.
func convInsecure(s string) (bool, error) {
s = strings.ToLower(s)
if s != "true" && s != "false" {
return false, fmt.Errorf("can't convert %q to bool", s)
}
return s == "true", nil
}
// loadInsecureFromEnvEndpoint returns a resolver that fetches
// insecure setting from envEndpoint is it possible.
func loadInsecureFromEnvEndpoint(envEndpoint []string) resolver[bool] {
return func(s setting[bool]) setting[bool] {
if s.Set {
// Passed, valid, options have precedence.
return s
}
for _, key := range envEndpoint {
if vStr := os.Getenv(key); vStr != "" {
u, err := url.Parse(vStr)
if err != nil {
otel.Handle(fmt.Errorf("invalid %s value %s: %w", key, vStr, err))
continue
}
return insecureFromScheme(s, u.Scheme)
}
}
return s
}
}
// convHeaders converts the OTel environment variable header value s into a
// mapping of header key to value. If s is invalid a partial result and error
// are returned.
func convHeaders(s string) (map[string]string, error) {
out := make(map[string]string)
var err error
for _, header := range strings.Split(s, ",") {
rawKey, rawVal, found := strings.Cut(header, "=")
if !found {
err = errors.Join(err, fmt.Errorf("invalid header: %s", header))
continue
}
escKey, e := url.PathUnescape(rawKey)
if e != nil {
err = errors.Join(err, fmt.Errorf("invalid header key: %s", rawKey))
continue
}
key := strings.TrimSpace(escKey)
escVal, e := url.PathUnescape(rawVal)
if e != nil {
err = errors.Join(err, fmt.Errorf("invalid header value: %s", rawVal))
continue
}
val := strings.TrimSpace(escVal)
out[key] = val
}
return out, err
}
// convDuration converts s into a duration of milliseconds. If s does not
// contain an integer, 0 and an error are returned.
func convDuration(s string) (time.Duration, error) {
d, err := strconv.Atoi(s)
if err != nil {
return 0, err
}
// OTel durations are defined in milliseconds.
return time.Duration(d) * time.Millisecond, nil
}
// loadEnvTLS returns a resolver that loads a *tls.Config from files defined by
// the OTLP TLS environment variables. This will load both the rootCAs and
// certificates used for mTLS.
//
// If the filepath defined is invalid or does not contain valid TLS files, an
// error is passed to the OTel ErrorHandler and no TLS configuration is
// provided.
func loadEnvTLS[T *tls.Config]() resolver[T] {
return func(s setting[T]) setting[T] {
if s.Set {
// Passed, valid, options have precedence.
return s
}
var rootCAs *x509.CertPool
var err error
for _, key := range envTLSCert {
if v := os.Getenv(key); v != "" {
rootCAs, err = loadCertPool(v)
break
}
}
var certs []tls.Certificate
for _, pair := range envTLSClient {
cert := os.Getenv(pair.Certificate)
key := os.Getenv(pair.Key)
if cert != "" && key != "" {
var e error
certs, e = loadCertificates(cert, key)
err = errors.Join(err, e)
break
}
}
if err != nil {
err = fmt.Errorf("failed to load TLS: %w", err)
otel.Handle(err)
} else if rootCAs != nil || certs != nil {
s.Set = true
s.Value = &tls.Config{RootCAs: rootCAs, Certificates: certs}
}
return s
}
}
// readFile is used for testing.
var readFile = os.ReadFile
// loadCertPool loads and returns the *x509.CertPool found at path if it exists
// and is valid. Otherwise, nil and an error is returned.
func loadCertPool(path string) (*x509.CertPool, error) {
b, err := readFile(path)
if err != nil {
return nil, err
}
cp := x509.NewCertPool()
if ok := cp.AppendCertsFromPEM(b); !ok {
return nil, errors.New("certificate not added")
}
return cp, nil
}
// loadCertificates loads and returns the tls.Certificate found at path if it
// exists and is valid. Otherwise, nil and an error is returned.
func loadCertificates(certPath, keyPath string) ([]tls.Certificate, error) {
cert, err := readFile(certPath)
if err != nil {
return nil, err
}
key, err := readFile(keyPath)
if err != nil {
return nil, err
}
crt, err := tls.X509KeyPair(cert, key)
if err != nil {
return nil, err
}
return []tls.Certificate{crt}, nil
}
// insecureFromScheme return setting if the connection should
// use client transport security or not.
// Empty scheme doesn't force insecure setting.
func insecureFromScheme(prev setting[bool], scheme string) setting[bool] {
if scheme == "https" {
return newSetting(false)
} else if len(scheme) > 0 {
return newSetting(true)
}
return prev
}
func compressorToCompression(compressor string) Compression {
c, err := convCompression(compressor)
if err != nil {
otel.Handle(fmt.Errorf("%w, using no compression as default", err))
return NoCompression
}
return c
}
// setting is a configuration setting value.
type setting[T any] struct {
Value T
Set bool
}
// newSetting returns a new setting with the value set.
func newSetting[T any](value T) setting[T] {
return setting[T]{Value: value, Set: true}
}
// resolver returns an updated setting after applying an resolution operation.
type resolver[T any] func(setting[T]) setting[T]
// Resolve returns a resolved version of s.
//
// It will apply all the passed fn in the order provided, chaining together the
// return setting to the next input. The setting s is used as the initial
// argument to the first fn.
//
// Each fn needs to validate if it should apply given the Set state of the
// setting. This will not perform any checks on the set state when chaining
// function.
func (s setting[T]) Resolve(fn ...resolver[T]) setting[T] {
for _, f := range fn {
s = f(s)
}
return s
}
// getEnv returns a resolver that will apply an environment variable value
// associated with the first set key to a setting value. The conv function is
// used to convert between the environment variable value and the setting type.
//
// If the input setting to the resolver is set, the environment variable will
// not be applied.
//
// Any error returned from conv is sent to the OTel ErrorHandler and the
// setting will not be updated.
func getEnv[T any](keys []string, conv func(string) (T, error)) resolver[T] {
return func(s setting[T]) setting[T] {
if s.Set {
// Passed, valid, options have precedence.
return s
}
for _, key := range keys {
if vStr := os.Getenv(key); vStr != "" {
v, err := conv(vStr)
if err == nil {
s.Value = v
s.Set = true
break
}
otel.Handle(fmt.Errorf("invalid %s value %s: %w", key, vStr, err))
}
}
return s
}
}
// fallback returns a resolve that will set a setting value to val if it is not
// already set.
//
// This is usually passed at the end of a resolver chain to ensure a default is
// applied if the setting has not already been set.
func fallback[T any](val T) resolver[T] {
return func(s setting[T]) setting[T] {
if !s.Set {
s.Value = val
s.Set = true
}
return s
}
}

View file

@ -0,0 +1,63 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
/*
Package otlploggrpc provides an OTLP log exporter using gRPC. The exporter uses gRPC to
transport OTLP protobuf payloads.
All Exporters must be created with [New].
The environment variables described below can be used for configuration.
OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT (default: "https://localhost:4317") -
target to which the exporter sends telemetry.
The target syntax is defined in https://github.com/grpc/grpc/blob/master/doc/naming.md.
The value must contain a scheme ("http" or "https") and host.
The value may additionally contain a port, and a path.
The value should not contain a query string or fragment.
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT takes precedence over OTEL_EXPORTER_OTLP_ENDPOINT.
The configuration can be overridden by [WithEndpoint], [WithEndpointURL], [WithInsecure], and [WithGRPCConn] options.
OTEL_EXPORTER_OTLP_INSECURE, OTEL_EXPORTER_OTLP_LOGS_INSECURE (default: "false") -
setting "true" disables client transport security for the exporter's gRPC connection.
You can use this only when an endpoint is provided without scheme.
OTEL_EXPORTER_OTLP_LOGS_INSECURE takes precedence over OTEL_EXPORTER_OTLP_INSECURE.
The configuration can be overridden by [WithInsecure], [WithGRPCConn] options.
OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_LOGS_HEADERS (default: none) -
key-value pairs used as gRPC metadata associated with gRPC requests.
The value is expected to be represented in a format matching the [W3C Baggage HTTP Header Content Format],
except that additional semi-colon delimited metadata is not supported.
Example value: "key1=value1,key2=value2".
OTEL_EXPORTER_OTLP_LOGS_HEADERS takes precedence over OTEL_EXPORTER_OTLP_HEADERS.
The configuration can be overridden by [WithHeaders] option.
OTEL_EXPORTER_OTLP_TIMEOUT, OTEL_EXPORTER_OTLP_LOGS_TIMEOUT (default: "10000") -
maximum time in milliseconds the OTLP exporter waits for each batch export.
OTEL_EXPORTER_OTLP_LOGS_TIMEOUT takes precedence over OTEL_EXPORTER_OTLP_TIMEOUT.
The configuration can be overridden by [WithTimeout] option.
OTEL_EXPORTER_OTLP_COMPRESSION, OTEL_EXPORTER_OTLP_LOGS_COMPRESSION (default: none) -
the gRPC compressor the exporter uses.
Supported value: "gzip".
OTEL_EXPORTER_OTLP_LOGS_COMPRESSION takes precedence over OTEL_EXPORTER_OTLP_COMPRESSION.
The configuration can be overridden by [WithCompressor], [WithGRPCConn] options.
OTEL_EXPORTER_OTLP_CERTIFICATE, OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE (default: none) -
the filepath to the trusted certificate to use when verifying a server's TLS credentials.
OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE takes precedence over OTEL_EXPORTER_OTLP_CERTIFICATE.
The configuration can be overridden by [WithTLSCredentials], [WithGRPCConn] options.
OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE, OTEL_EXPORTER_OTLP_LOGS_CLIENT_CERTIFICATE (default: none) -
the filepath to the client certificate/chain trust for client's private key to use in mTLS communication in PEM format.
OTEL_EXPORTER_OTLP_LOGS_CLIENT_CERTIFICATE takes precedence over OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE.
The configuration can be overridden by [WithTLSCredentials], [WithGRPCConn] options.
OTEL_EXPORTER_OTLP_CLIENT_KEY, OTEL_EXPORTER_OTLP_LOGS_CLIENT_KEY (default: none) -
the filepath to the client's private key to use in mTLS communication in PEM format.
OTEL_EXPORTER_OTLP_LOGS_CLIENT_KEY takes precedence over OTEL_EXPORTER_OTLP_CLIENT_KEY.
The configuration can be overridden by [WithTLSCredentials], [WithGRPCConn] option.
[W3C Baggage HTTP Header Content Format]: https://www.w3.org/TR/baggage/#header-content
*/
package otlploggrpc // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc"

View file

@ -0,0 +1,93 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploggrpc // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc"
import (
"context"
"sync"
"sync/atomic"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/transform"
"go.opentelemetry.io/otel/sdk/log"
logpb "go.opentelemetry.io/proto/otlp/logs/v1"
)
type logClient interface {
UploadLogs(ctx context.Context, rl []*logpb.ResourceLogs) error
Shutdown(context.Context) error
}
// Exporter is a OpenTelemetry log Exporter. It transports log data encoded as
// OTLP protobufs using gRPC.
// All Exporters must be created with [New].
type Exporter struct {
// Ensure synchronous access to the client across all functionality.
clientMu sync.Mutex
client logClient
stopped atomic.Bool
}
// Compile-time check Exporter implements [log.Exporter].
var _ log.Exporter = (*Exporter)(nil)
// New returns a new [Exporter].
//
// It is recommended to use it with a [BatchProcessor]
// or other processor exporting records asynchronously.
func New(_ context.Context, options ...Option) (*Exporter, error) {
cfg := newConfig(options)
c, err := newClient(cfg)
if err != nil {
return nil, err
}
return newExporter(c), nil
}
func newExporter(c logClient) *Exporter {
var e Exporter
e.client = c
return &e
}
var transformResourceLogs = transform.ResourceLogs
// Export transforms and transmits log records to an OTLP receiver.
//
// This method returns nil and drops records if called after Shutdown.
// This method returns an error if the method is canceled by the passed context.
func (e *Exporter) Export(ctx context.Context, records []log.Record) error {
if e.stopped.Load() {
return nil
}
otlp := transformResourceLogs(records)
if otlp == nil {
return nil
}
e.clientMu.Lock()
defer e.clientMu.Unlock()
return e.client.UploadLogs(ctx, otlp)
}
// Shutdown shuts down the Exporter. Calls to Export or ForceFlush will perform
// no operation after this is called.
func (e *Exporter) Shutdown(ctx context.Context) error {
if e.stopped.Swap(true) {
return nil
}
e.clientMu.Lock()
defer e.clientMu.Unlock()
err := e.client.Shutdown(ctx)
e.client = newNoopClient()
return err
}
// ForceFlush does nothing. The Exporter holds no state.
func (e *Exporter) ForceFlush(ctx context.Context) error {
return nil
}

View file

@ -0,0 +1,145 @@
// Code created by gotmpl. DO NOT MODIFY.
// source: internal/shared/otlp/retry/retry.go.tmpl
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
// Package retry provides request retry functionality that can perform
// configurable exponential backoff for transient errors and honor any
// explicit throttle responses received.
package retry // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/retry"
import (
"context"
"fmt"
"time"
"github.com/cenkalti/backoff/v4"
)
// DefaultConfig are the recommended defaults to use.
var DefaultConfig = Config{
Enabled: true,
InitialInterval: 5 * time.Second,
MaxInterval: 30 * time.Second,
MaxElapsedTime: time.Minute,
}
// Config defines configuration for retrying batches in case of export failure
// using an exponential backoff.
type Config struct {
// Enabled indicates whether to not retry sending batches in case of
// export failure.
Enabled bool
// InitialInterval the time to wait after the first failure before
// retrying.
InitialInterval time.Duration
// MaxInterval is the upper bound on backoff interval. Once this value is
// reached the delay between consecutive retries will always be
// `MaxInterval`.
MaxInterval time.Duration
// MaxElapsedTime is the maximum amount of time (including retries) spent
// trying to send a request/batch. Once this value is reached, the data
// is discarded.
MaxElapsedTime time.Duration
}
// RequestFunc wraps a request with retry logic.
type RequestFunc func(context.Context, func(context.Context) error) error
// EvaluateFunc returns if an error is retry-able and if an explicit throttle
// duration should be honored that was included in the error.
//
// The function must return true if the error argument is retry-able,
// otherwise it must return false for the first return parameter.
//
// The function must return a non-zero time.Duration if the error contains
// explicit throttle duration that should be honored, otherwise it must return
// a zero valued time.Duration.
type EvaluateFunc func(error) (bool, time.Duration)
// RequestFunc returns a RequestFunc using the evaluate function to determine
// if requests can be retried and based on the exponential backoff
// configuration of c.
func (c Config) RequestFunc(evaluate EvaluateFunc) RequestFunc {
if !c.Enabled {
return func(ctx context.Context, fn func(context.Context) error) error {
return fn(ctx)
}
}
return func(ctx context.Context, fn func(context.Context) error) error {
// Do not use NewExponentialBackOff since it calls Reset and the code here
// must call Reset after changing the InitialInterval (this saves an
// unnecessary call to Now).
b := &backoff.ExponentialBackOff{
InitialInterval: c.InitialInterval,
RandomizationFactor: backoff.DefaultRandomizationFactor,
Multiplier: backoff.DefaultMultiplier,
MaxInterval: c.MaxInterval,
MaxElapsedTime: c.MaxElapsedTime,
Stop: backoff.Stop,
Clock: backoff.SystemClock,
}
b.Reset()
for {
err := fn(ctx)
if err == nil {
return nil
}
retryable, throttle := evaluate(err)
if !retryable {
return err
}
bOff := b.NextBackOff()
if bOff == backoff.Stop {
return fmt.Errorf("max retry time elapsed: %w", err)
}
// Wait for the greater of the backoff or throttle delay.
var delay time.Duration
if bOff > throttle {
delay = bOff
} else {
elapsed := b.GetElapsedTime()
if b.MaxElapsedTime != 0 && elapsed+throttle > b.MaxElapsedTime {
return fmt.Errorf("max retry time would elapse: %w", err)
}
delay = throttle
}
if ctxErr := waitFunc(ctx, delay); ctxErr != nil {
return fmt.Errorf("%w: %w", ctxErr, err)
}
}
}
}
// Allow override for testing.
var waitFunc = wait
// wait takes the caller's context, and the amount of time to wait. It will
// return nil if the timer fires before or at the same time as the context's
// deadline. This indicates that the call can be retried.
func wait(ctx context.Context, delay time.Duration) error {
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
// Handle the case where the timer and context deadline end
// simultaneously by prioritizing the timer expiration nil value
// response.
select {
case <-timer.C:
default:
return ctx.Err()
}
case <-timer.C:
}
return nil
}

View file

@ -0,0 +1,391 @@
// Code created by gotmpl. DO NOT MODIFY.
// source: internal/shared/otlp/otlplog/transform/log.go.tmpl
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
// Package transform provides transformation functionality from the
// sdk/log data-types into OTLP data-types.
package transform // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/transform"
import (
"time"
cpb "go.opentelemetry.io/proto/otlp/common/v1"
lpb "go.opentelemetry.io/proto/otlp/logs/v1"
rpb "go.opentelemetry.io/proto/otlp/resource/v1"
"go.opentelemetry.io/otel/attribute"
api "go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/sdk/instrumentation"
"go.opentelemetry.io/otel/sdk/log"
)
// ResourceLogs returns an slice of OTLP ResourceLogs generated from records.
func ResourceLogs(records []log.Record) []*lpb.ResourceLogs {
if len(records) == 0 {
return nil
}
resMap := make(map[attribute.Distinct]*lpb.ResourceLogs)
type key struct {
r attribute.Distinct
is instrumentation.Scope
}
scopeMap := make(map[key]*lpb.ScopeLogs)
var resources int
for _, r := range records {
res := r.Resource()
rKey := res.Equivalent()
scope := r.InstrumentationScope()
k := key{
r: rKey,
is: scope,
}
sl, iOk := scopeMap[k]
if !iOk {
sl = new(lpb.ScopeLogs)
var emptyScope instrumentation.Scope
if scope != emptyScope {
sl.Scope = &cpb.InstrumentationScope{
Name: scope.Name,
Version: scope.Version,
Attributes: AttrIter(scope.Attributes.Iter()),
}
sl.SchemaUrl = scope.SchemaURL
}
scopeMap[k] = sl
}
sl.LogRecords = append(sl.LogRecords, LogRecord(r))
rl, rOk := resMap[rKey]
if !rOk {
resources++
rl = new(lpb.ResourceLogs)
if res.Len() > 0 {
rl.Resource = &rpb.Resource{
Attributes: AttrIter(res.Iter()),
}
}
rl.SchemaUrl = res.SchemaURL()
resMap[rKey] = rl
}
if !iOk {
rl.ScopeLogs = append(rl.ScopeLogs, sl)
}
}
// Transform the categorized map into a slice
resLogs := make([]*lpb.ResourceLogs, 0, resources)
for _, rl := range resMap {
resLogs = append(resLogs, rl)
}
return resLogs
}
// LogRecord returns an OTLP LogRecord generated from record.
func LogRecord(record log.Record) *lpb.LogRecord {
r := &lpb.LogRecord{
TimeUnixNano: timeUnixNano(record.Timestamp()),
ObservedTimeUnixNano: timeUnixNano(record.ObservedTimestamp()),
EventName: record.EventName(),
SeverityNumber: SeverityNumber(record.Severity()),
SeverityText: record.SeverityText(),
Body: LogAttrValue(record.Body()),
Attributes: make([]*cpb.KeyValue, 0, record.AttributesLen()),
Flags: uint32(record.TraceFlags()),
// TODO: DroppedAttributesCount: /* ... */,
}
record.WalkAttributes(func(kv api.KeyValue) bool {
r.Attributes = append(r.Attributes, LogAttr(kv))
return true
})
if tID := record.TraceID(); tID.IsValid() {
r.TraceId = tID[:]
}
if sID := record.SpanID(); sID.IsValid() {
r.SpanId = sID[:]
}
return r
}
// timeUnixNano returns t as a Unix time, the number of nanoseconds elapsed
// since January 1, 1970 UTC as uint64. The result is undefined if the Unix
// time in nanoseconds cannot be represented by an int64 (a date before the
// year 1678 or after 2262). timeUnixNano on the zero Time returns 0. The
// result does not depend on the location associated with t.
func timeUnixNano(t time.Time) uint64 {
nano := t.UnixNano()
if nano < 0 {
return 0
}
return uint64(nano) // nolint:gosec // Overflow checked.
}
// AttrIter transforms an [attribute.Iterator] into OTLP key-values.
func AttrIter(iter attribute.Iterator) []*cpb.KeyValue {
l := iter.Len()
if l == 0 {
return nil
}
out := make([]*cpb.KeyValue, 0, l)
for iter.Next() {
out = append(out, Attr(iter.Attribute()))
}
return out
}
// Attrs transforms a slice of [attribute.KeyValue] into OTLP key-values.
func Attrs(attrs []attribute.KeyValue) []*cpb.KeyValue {
if len(attrs) == 0 {
return nil
}
out := make([]*cpb.KeyValue, 0, len(attrs))
for _, kv := range attrs {
out = append(out, Attr(kv))
}
return out
}
// Attr transforms an [attribute.KeyValue] into an OTLP key-value.
func Attr(kv attribute.KeyValue) *cpb.KeyValue {
return &cpb.KeyValue{Key: string(kv.Key), Value: AttrValue(kv.Value)}
}
// AttrValue transforms an [attribute.Value] into an OTLP AnyValue.
func AttrValue(v attribute.Value) *cpb.AnyValue {
av := new(cpb.AnyValue)
switch v.Type() {
case attribute.BOOL:
av.Value = &cpb.AnyValue_BoolValue{
BoolValue: v.AsBool(),
}
case attribute.BOOLSLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: boolSliceValues(v.AsBoolSlice()),
},
}
case attribute.INT64:
av.Value = &cpb.AnyValue_IntValue{
IntValue: v.AsInt64(),
}
case attribute.INT64SLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: int64SliceValues(v.AsInt64Slice()),
},
}
case attribute.FLOAT64:
av.Value = &cpb.AnyValue_DoubleValue{
DoubleValue: v.AsFloat64(),
}
case attribute.FLOAT64SLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: float64SliceValues(v.AsFloat64Slice()),
},
}
case attribute.STRING:
av.Value = &cpb.AnyValue_StringValue{
StringValue: v.AsString(),
}
case attribute.STRINGSLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: stringSliceValues(v.AsStringSlice()),
},
}
default:
av.Value = &cpb.AnyValue_StringValue{
StringValue: "INVALID",
}
}
return av
}
func boolSliceValues(vals []bool) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_BoolValue{
BoolValue: v,
},
}
}
return converted
}
func int64SliceValues(vals []int64) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_IntValue{
IntValue: v,
},
}
}
return converted
}
func float64SliceValues(vals []float64) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_DoubleValue{
DoubleValue: v,
},
}
}
return converted
}
func stringSliceValues(vals []string) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_StringValue{
StringValue: v,
},
}
}
return converted
}
// Attrs transforms a slice of [api.KeyValue] into OTLP key-values.
func LogAttrs(attrs []api.KeyValue) []*cpb.KeyValue {
if len(attrs) == 0 {
return nil
}
out := make([]*cpb.KeyValue, 0, len(attrs))
for _, kv := range attrs {
out = append(out, LogAttr(kv))
}
return out
}
// LogAttr transforms an [api.KeyValue] into an OTLP key-value.
func LogAttr(attr api.KeyValue) *cpb.KeyValue {
return &cpb.KeyValue{
Key: attr.Key,
Value: LogAttrValue(attr.Value),
}
}
// LogAttrValues transforms a slice of [api.Value] into an OTLP []AnyValue.
func LogAttrValues(vals []api.Value) []*cpb.AnyValue {
if len(vals) == 0 {
return nil
}
out := make([]*cpb.AnyValue, 0, len(vals))
for _, v := range vals {
out = append(out, LogAttrValue(v))
}
return out
}
// LogAttrValue transforms an [api.Value] into an OTLP AnyValue.
func LogAttrValue(v api.Value) *cpb.AnyValue {
av := new(cpb.AnyValue)
switch v.Kind() {
case api.KindBool:
av.Value = &cpb.AnyValue_BoolValue{
BoolValue: v.AsBool(),
}
case api.KindInt64:
av.Value = &cpb.AnyValue_IntValue{
IntValue: v.AsInt64(),
}
case api.KindFloat64:
av.Value = &cpb.AnyValue_DoubleValue{
DoubleValue: v.AsFloat64(),
}
case api.KindString:
av.Value = &cpb.AnyValue_StringValue{
StringValue: v.AsString(),
}
case api.KindBytes:
av.Value = &cpb.AnyValue_BytesValue{
BytesValue: v.AsBytes(),
}
case api.KindSlice:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: LogAttrValues(v.AsSlice()),
},
}
case api.KindMap:
av.Value = &cpb.AnyValue_KvlistValue{
KvlistValue: &cpb.KeyValueList{
Values: LogAttrs(v.AsMap()),
},
}
default:
av.Value = &cpb.AnyValue_StringValue{
StringValue: "INVALID",
}
}
return av
}
// SeverityNumber transforms a [log.Severity] into an OTLP SeverityNumber.
func SeverityNumber(s api.Severity) lpb.SeverityNumber {
switch s {
case api.SeverityTrace:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE
case api.SeverityTrace2:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE2
case api.SeverityTrace3:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE3
case api.SeverityTrace4:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE4
case api.SeverityDebug:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG
case api.SeverityDebug2:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG2
case api.SeverityDebug3:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG3
case api.SeverityDebug4:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG4
case api.SeverityInfo:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO
case api.SeverityInfo2:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO2
case api.SeverityInfo3:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO3
case api.SeverityInfo4:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO4
case api.SeverityWarn:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN
case api.SeverityWarn2:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN2
case api.SeverityWarn3:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN3
case api.SeverityWarn4:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN4
case api.SeverityError:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR
case api.SeverityError2:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR2
case api.SeverityError3:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR3
case api.SeverityError4:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR4
case api.SeverityFatal:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL
case api.SeverityFatal2:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL2
case api.SeverityFatal3:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL3
case api.SeverityFatal4:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL4
}
return lpb.SeverityNumber_SEVERITY_NUMBER_UNSPECIFIED
}

View file

@ -0,0 +1,9 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploggrpc // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc"
// Version is the current release version of the OpenTelemetry OTLP over gRPC logs exporter in use.
func Version() string {
return "0.11.0"
}

View file

@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View file

@ -0,0 +1,3 @@
# OTLP Log HTTP Exporter
[![PkgGoDev](https://pkg.go.dev/badge/go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp)](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp)

View file

@ -0,0 +1,343 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploghttp // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
import (
"bytes"
"compress/gzip"
"context"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
"google.golang.org/protobuf/proto"
"go.opentelemetry.io/otel"
collogpb "go.opentelemetry.io/proto/otlp/collector/logs/v1"
logpb "go.opentelemetry.io/proto/otlp/logs/v1"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp/internal/retry"
)
type client struct {
uploadLogs func(context.Context, []*logpb.ResourceLogs) error
}
func (c *client) UploadLogs(ctx context.Context, rl []*logpb.ResourceLogs) error {
if c.uploadLogs != nil {
return c.uploadLogs(ctx, rl)
}
return nil
}
func newNoopClient() *client {
return &client{}
}
// newHTTPClient creates a new HTTP log client.
func newHTTPClient(cfg config) (*client, error) {
hc := &http.Client{
Transport: ourTransport,
Timeout: cfg.timeout.Value,
}
if cfg.tlsCfg.Value != nil || cfg.proxy.Value != nil {
clonedTransport := ourTransport.Clone()
hc.Transport = clonedTransport
if cfg.tlsCfg.Value != nil {
clonedTransport.TLSClientConfig = cfg.tlsCfg.Value
}
if cfg.proxy.Value != nil {
clonedTransport.Proxy = cfg.proxy.Value
}
}
u := &url.URL{
Scheme: "https",
Host: cfg.endpoint.Value,
Path: cfg.path.Value,
}
if cfg.insecure.Value {
u.Scheme = "http"
}
// Body is set when this is cloned during upload.
req, err := http.NewRequest(http.MethodPost, u.String(), http.NoBody)
if err != nil {
return nil, err
}
userAgent := "OTel Go OTLP over HTTP/protobuf logs exporter/" + Version()
req.Header.Set("User-Agent", userAgent)
if n := len(cfg.headers.Value); n > 0 {
for k, v := range cfg.headers.Value {
req.Header.Set(k, v)
}
}
req.Header.Set("Content-Type", "application/x-protobuf")
c := &httpClient{
compression: cfg.compression.Value,
req: req,
requestFunc: cfg.retryCfg.Value.RequestFunc(evaluate),
client: hc,
}
return &client{uploadLogs: c.uploadLogs}, nil
}
type httpClient struct {
// req is cloned for every upload the client makes.
req *http.Request
compression Compression
requestFunc retry.RequestFunc
client *http.Client
}
// Keep it in sync with golang's DefaultTransport from net/http! We
// have our own copy to avoid handling a situation where the
// DefaultTransport is overwritten with some different implementation
// of http.RoundTripper or it's modified by another package.
var ourTransport = &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 100,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
}
func (c *httpClient) uploadLogs(ctx context.Context, data []*logpb.ResourceLogs) error {
// The Exporter synchronizes access to client methods. This is not called
// after the Exporter is shutdown. Only thing to do here is send data.
pbRequest := &collogpb.ExportLogsServiceRequest{ResourceLogs: data}
body, err := proto.Marshal(pbRequest)
if err != nil {
return err
}
request, err := c.newRequest(ctx, body)
if err != nil {
return err
}
return c.requestFunc(ctx, func(iCtx context.Context) error {
select {
case <-iCtx.Done():
return iCtx.Err()
default:
}
request.reset(iCtx)
resp, err := c.client.Do(request.Request)
var urlErr *url.Error
if errors.As(err, &urlErr) && urlErr.Temporary() {
return newResponseError(http.Header{}, err)
}
if err != nil {
return err
}
if resp != nil && resp.Body != nil {
defer func() {
if err := resp.Body.Close(); err != nil {
otel.Handle(err)
}
}()
}
if sc := resp.StatusCode; sc >= 200 && sc <= 299 {
// Success, do not retry.
// Read the partial success message, if any.
var respData bytes.Buffer
if _, err := io.Copy(&respData, resp.Body); err != nil {
return err
}
if respData.Len() == 0 {
return nil
}
if resp.Header.Get("Content-Type") == "application/x-protobuf" {
var respProto collogpb.ExportLogsServiceResponse
if err := proto.Unmarshal(respData.Bytes(), &respProto); err != nil {
return err
}
if respProto.PartialSuccess != nil {
msg := respProto.PartialSuccess.GetErrorMessage()
n := respProto.PartialSuccess.GetRejectedLogRecords()
if n != 0 || msg != "" {
err := fmt.Errorf("OTLP partial success: %s (%d log records rejected)", msg, n)
otel.Handle(err)
}
}
}
return nil
}
// Error cases.
// server may return a message with the response
// body, so we read it to include in the error
// message to be returned. It will help in
// debugging the actual issue.
var respData bytes.Buffer
if _, err := io.Copy(&respData, resp.Body); err != nil {
return err
}
respStr := strings.TrimSpace(respData.String())
if len(respStr) == 0 {
respStr = "(empty)"
}
bodyErr := fmt.Errorf("body: %s", respStr)
switch resp.StatusCode {
case http.StatusTooManyRequests,
http.StatusBadGateway,
http.StatusServiceUnavailable,
http.StatusGatewayTimeout:
// Retryable failure.
return newResponseError(resp.Header, bodyErr)
default:
// Non-retryable failure.
return fmt.Errorf("failed to send logs to %s: %s (%w)", request.URL, resp.Status, bodyErr)
}
})
}
var gzPool = sync.Pool{
New: func() interface{} {
w := gzip.NewWriter(io.Discard)
return w
},
}
func (c *httpClient) newRequest(ctx context.Context, body []byte) (request, error) {
r := c.req.Clone(ctx)
req := request{Request: r}
switch c.compression {
case NoCompression:
r.ContentLength = (int64)(len(body))
req.bodyReader = bodyReader(body)
case GzipCompression:
// Ensure the content length is not used.
r.ContentLength = -1
r.Header.Set("Content-Encoding", "gzip")
gz := gzPool.Get().(*gzip.Writer)
defer gzPool.Put(gz)
var b bytes.Buffer
gz.Reset(&b)
if _, err := gz.Write(body); err != nil {
return req, err
}
// Close needs to be called to ensure body is fully written.
if err := gz.Close(); err != nil {
return req, err
}
req.bodyReader = bodyReader(b.Bytes())
}
return req, nil
}
// bodyReader returns a closure returning a new reader for buf.
func bodyReader(buf []byte) func() io.ReadCloser {
return func() io.ReadCloser {
return io.NopCloser(bytes.NewReader(buf))
}
}
// request wraps an http.Request with a resettable body reader.
type request struct {
*http.Request
// bodyReader allows the same body to be used for multiple requests.
bodyReader func() io.ReadCloser
}
// reset reinitializes the request Body and uses ctx for the request.
func (r *request) reset(ctx context.Context) {
r.Body = r.bodyReader()
r.Request = r.WithContext(ctx)
}
// retryableError represents a request failure that can be retried.
type retryableError struct {
throttle int64
err error
}
// newResponseError returns a retryableError and will extract any explicit
// throttle delay contained in headers. The returned error wraps wrapped
// if it is not nil.
func newResponseError(header http.Header, wrapped error) error {
var rErr retryableError
if v := header.Get("Retry-After"); v != "" {
if t, err := strconv.ParseInt(v, 10, 64); err == nil {
rErr.throttle = t
}
}
rErr.err = wrapped
return rErr
}
func (e retryableError) Error() string {
if e.err != nil {
return fmt.Sprintf("retry-able request failure: %v", e.err.Error())
}
return "retry-able request failure"
}
func (e retryableError) Unwrap() error {
return e.err
}
func (e retryableError) As(target interface{}) bool {
if e.err == nil {
return false
}
switch v := target.(type) {
case **retryableError:
*v = &e
return true
default:
return false
}
}
// evaluate returns if err is retry-able. If it is and it includes an explicit
// throttling delay, that delay is also returned.
func evaluate(err error) (bool, time.Duration) {
if err == nil {
return false, 0
}
// Do not use errors.As here, this should only be flattened one layer. If
// there are several chained errors, all the errors above it will be
// discarded if errors.As is used instead.
rErr, ok := err.(retryableError) //nolint:errorlint
if !ok {
return false, 0
}
return true, time.Duration(rErr.throttle)
}

View file

@ -0,0 +1,602 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploghttp // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
import (
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp/internal/retry"
"go.opentelemetry.io/otel/internal/global"
)
// Default values.
var (
defaultEndpoint = "localhost:4318"
defaultPath = "/v1/logs"
defaultTimeout = 10 * time.Second
defaultProxy HTTPTransportProxyFunc = http.ProxyFromEnvironment
defaultRetryCfg = retry.DefaultConfig
)
// Environment variable keys.
var (
envEndpoint = []string{
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
"OTEL_EXPORTER_OTLP_ENDPOINT",
}
envInsecure = envEndpoint
// Split because these are parsed differently.
envPathSignal = []string{"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT"}
envPathOTLP = []string{"OTEL_EXPORTER_OTLP_ENDPOINT"}
envHeaders = []string{
"OTEL_EXPORTER_OTLP_LOGS_HEADERS",
"OTEL_EXPORTER_OTLP_HEADERS",
}
envCompression = []string{
"OTEL_EXPORTER_OTLP_LOGS_COMPRESSION",
"OTEL_EXPORTER_OTLP_COMPRESSION",
}
envTimeout = []string{
"OTEL_EXPORTER_OTLP_LOGS_TIMEOUT",
"OTEL_EXPORTER_OTLP_TIMEOUT",
}
envTLSCert = []string{
"OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE",
"OTEL_EXPORTER_OTLP_CERTIFICATE",
}
envTLSClient = []struct {
Certificate string
Key string
}{
{
"OTEL_EXPORTER_OTLP_LOGS_CLIENT_CERTIFICATE",
"OTEL_EXPORTER_OTLP_LOGS_CLIENT_KEY",
},
{
"OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE",
"OTEL_EXPORTER_OTLP_CLIENT_KEY",
},
}
)
// Option applies an option to the Exporter.
type Option interface {
applyHTTPOption(config) config
}
type fnOpt func(config) config
func (f fnOpt) applyHTTPOption(c config) config { return f(c) }
type config struct {
endpoint setting[string]
path setting[string]
insecure setting[bool]
tlsCfg setting[*tls.Config]
headers setting[map[string]string]
compression setting[Compression]
timeout setting[time.Duration]
proxy setting[HTTPTransportProxyFunc]
retryCfg setting[retry.Config]
}
func newConfig(options []Option) config {
var c config
for _, opt := range options {
c = opt.applyHTTPOption(c)
}
c.endpoint = c.endpoint.Resolve(
getenv[string](envEndpoint, convEndpoint),
fallback[string](defaultEndpoint),
)
c.path = c.path.Resolve(
getenv[string](envPathSignal, convPathExact),
getenv[string](envPathOTLP, convPath),
fallback[string](defaultPath),
)
c.insecure = c.insecure.Resolve(
getenv[bool](envInsecure, convInsecure),
)
c.tlsCfg = c.tlsCfg.Resolve(
loadEnvTLS[*tls.Config](),
)
c.headers = c.headers.Resolve(
getenv[map[string]string](envHeaders, convHeaders),
)
c.compression = c.compression.Resolve(
getenv[Compression](envCompression, convCompression),
)
c.timeout = c.timeout.Resolve(
getenv[time.Duration](envTimeout, convDuration),
fallback[time.Duration](defaultTimeout),
)
c.proxy = c.proxy.Resolve(
fallback[HTTPTransportProxyFunc](defaultProxy),
)
c.retryCfg = c.retryCfg.Resolve(
fallback[retry.Config](defaultRetryCfg),
)
return c
}
// WithEndpoint sets the target endpoint the Exporter will connect to. This
// endpoint is specified as a host and optional port, no path or scheme should
// be included (see WithInsecure and WithURLPath).
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, that variable
// value will be used. If both environment variables are set,
// OTEL_EXPORTER_OTLP_LOGS_ENDPOINT will take precedence. If an environment
// variable is set, and this option is passed, this option will take precedence.
//
// If both this option and WithEndpointURL are used, the last used option will
// take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, "localhost:4318" will be used.
func WithEndpoint(endpoint string) Option {
return fnOpt(func(c config) config {
c.endpoint = newSetting(endpoint)
return c
})
}
// WithEndpointURL sets the target endpoint URL the Exporter will connect to.
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, that variable
// value will be used. If both environment variables are set,
// OTEL_EXPORTER_OTLP_LOGS_ENDPOINT will take precedence. If an environment
// variable is set, and this option is passed, this option will take precedence.
//
// If both this option and WithEndpoint are used, the last used option will
// take precedence.
//
// If an invalid URL is provided, the default value will be kept.
//
// By default, if an environment variable is not set, and this option is not
// passed, "localhost:4318" will be used.
func WithEndpointURL(rawURL string) Option {
u, err := url.Parse(rawURL)
if err != nil {
global.Error(err, "otlplog: parse endpoint url", "url", rawURL)
return fnOpt(func(c config) config { return c })
}
return fnOpt(func(c config) config {
c.endpoint = newSetting(u.Host)
c.path = newSetting(u.Path)
c.insecure = newSetting(u.Scheme != "https")
return c
})
}
// Compression describes the compression used for exported payloads.
type Compression int
const (
// NoCompression represents that no compression should be used.
NoCompression Compression = iota
// GzipCompression represents that gzip compression should be used.
GzipCompression
)
// WithCompression sets the compression strategy the Exporter will use to
// compress the HTTP body.
//
// If the OTEL_EXPORTER_OTLP_COMPRESSION or
// OTEL_EXPORTER_OTLP_LOGS_COMPRESSION environment variable is set, and
// this option is not passed, that variable value will be used. That value can
// be either "none" or "gzip". If both are set,
// OTEL_EXPORTER_OTLP_LOGS_COMPRESSION will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, no compression strategy will be used.
func WithCompression(compression Compression) Option {
return fnOpt(func(c config) config {
c.compression = newSetting(compression)
return c
})
}
// WithURLPath sets the URL path the Exporter will send requests to.
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, the path
// contained in that variable value will be used. If both are set,
// OTEL_EXPORTER_OTLP_LOGS_ENDPOINT will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, "/v1/logs" will be used.
func WithURLPath(urlPath string) Option {
return fnOpt(func(c config) config {
c.path = newSetting(urlPath)
return c
})
}
// WithTLSClientConfig sets the TLS configuration the Exporter will use for
// HTTP requests.
//
// If the OTEL_EXPORTER_OTLP_CERTIFICATE or
// OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE environment variable is set, and
// this option is not passed, that variable value will be used. The value will
// be parsed the filepath of the TLS certificate chain to use. If both are
// set, OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, the system default configuration is used.
func WithTLSClientConfig(tlsCfg *tls.Config) Option {
return fnOpt(func(c config) config {
c.tlsCfg = newSetting(tlsCfg.Clone())
return c
})
}
// WithInsecure disables client transport security for the Exporter's HTTP
// connection.
//
// If the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
// environment variable is set, and this option is not passed, that variable
// value will be used to determine client security. If the endpoint has a
// scheme of "http" or "unix" client security will be disabled. If both are
// set, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, client security will be used.
func WithInsecure() Option {
return fnOpt(func(c config) config {
c.insecure = newSetting(true)
return c
})
}
// WithHeaders will send the provided headers with each HTTP requests.
//
// If the OTEL_EXPORTER_OTLP_HEADERS or OTEL_EXPORTER_OTLP_LOGS_HEADERS
// environment variable is set, and this option is not passed, that variable
// value will be used. The value will be parsed as a list of key value pairs.
// These pairs are expected to be in the W3C Correlation-Context format
// without additional semi-colon delimited metadata (i.e. "k1=v1,k2=v2"). If
// both are set, OTEL_EXPORTER_OTLP_LOGS_HEADERS will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, no user headers will be set.
func WithHeaders(headers map[string]string) Option {
return fnOpt(func(c config) config {
c.headers = newSetting(headers)
return c
})
}
// WithTimeout sets the max amount of time an Exporter will attempt an export.
//
// This takes precedence over any retry settings defined by WithRetry. Once
// this time limit has been reached the export is abandoned and the log data is
// dropped.
//
// If the OTEL_EXPORTER_OTLP_TIMEOUT or OTEL_EXPORTER_OTLP_LOGS_TIMEOUT
// environment variable is set, and this option is not passed, that variable
// value will be used. The value will be parsed as an integer representing the
// timeout in milliseconds. If both are set,
// OTEL_EXPORTER_OTLP_LOGS_TIMEOUT will take precedence.
//
// By default, if an environment variable is not set, and this option is not
// passed, a timeout of 10 seconds will be used.
func WithTimeout(duration time.Duration) Option {
return fnOpt(func(c config) config {
c.timeout = newSetting(duration)
return c
})
}
// RetryConfig defines configuration for retrying the export of log data that
// failed.
type RetryConfig retry.Config
// WithRetry sets the retry policy for transient retryable errors that are
// returned by the target endpoint.
//
// If the target endpoint responds with not only a retryable error, but
// explicitly returns a backoff time in the response, that time will take
// precedence over these settings.
//
// If unset, the default retry policy will be used. It will retry the export
// 5 seconds after receiving a retryable error and increase exponentially
// after each error for no more than a total time of 1 minute.
func WithRetry(rc RetryConfig) Option {
return fnOpt(func(c config) config {
c.retryCfg = newSetting(retry.Config(rc))
return c
})
}
// HTTPTransportProxyFunc is a function that resolves which URL to use as proxy
// for a given request. This type is compatible with http.Transport.Proxy and
// can be used to set a custom proxy function to the OTLP HTTP client.
type HTTPTransportProxyFunc func(*http.Request) (*url.URL, error)
// WithProxy sets the Proxy function the client will use to determine the
// proxy to use for an HTTP request. If this option is not used, the client
// will use [http.ProxyFromEnvironment].
func WithProxy(pf HTTPTransportProxyFunc) Option {
return fnOpt(func(c config) config {
c.proxy = newSetting(pf)
return c
})
}
// setting is a configuration setting value.
type setting[T any] struct {
Value T
Set bool
}
// newSetting returns a new setting with the value set.
func newSetting[T any](value T) setting[T] {
return setting[T]{Value: value, Set: true}
}
// resolver returns an updated setting after applying an resolution operation.
type resolver[T any] func(setting[T]) setting[T]
// Resolve returns a resolved version of s.
//
// It will apply all the passed fn in the order provided, chaining together the
// return setting to the next input. The setting s is used as the initial
// argument to the first fn.
//
// Each fn needs to validate if it should apply given the Set state of the
// setting. This will not perform any checks on the set state when chaining
// function.
func (s setting[T]) Resolve(fn ...resolver[T]) setting[T] {
for _, f := range fn {
s = f(s)
}
return s
}
// loadEnvTLS returns a resolver that loads a *tls.Config from files defined by
// the OTLP TLS environment variables. This will load both the rootCAs and
// certificates used for mTLS.
//
// If the filepath defined is invalid or does not contain valid TLS files, an
// error is passed to the OTel ErrorHandler and no TLS configuration is
// provided.
func loadEnvTLS[T *tls.Config]() resolver[T] {
return func(s setting[T]) setting[T] {
if s.Set {
// Passed, valid, options have precedence.
return s
}
var rootCAs *x509.CertPool
var err error
for _, key := range envTLSCert {
if v := os.Getenv(key); v != "" {
rootCAs, err = loadCertPool(v)
break
}
}
var certs []tls.Certificate
for _, pair := range envTLSClient {
cert := os.Getenv(pair.Certificate)
key := os.Getenv(pair.Key)
if cert != "" && key != "" {
var e error
certs, e = loadCertificates(cert, key)
err = errors.Join(err, e)
break
}
}
if err != nil {
err = fmt.Errorf("failed to load TLS: %w", err)
otel.Handle(err)
} else if rootCAs != nil || certs != nil {
s.Set = true
s.Value = &tls.Config{RootCAs: rootCAs, Certificates: certs}
}
return s
}
}
// readFile is used for testing.
var readFile = os.ReadFile
// loadCertPool loads and returns the *x509.CertPool found at path if it exists
// and is valid. Otherwise, nil and an error is returned.
func loadCertPool(path string) (*x509.CertPool, error) {
b, err := readFile(path)
if err != nil {
return nil, err
}
cp := x509.NewCertPool()
if ok := cp.AppendCertsFromPEM(b); !ok {
return nil, errors.New("certificate not added")
}
return cp, nil
}
// loadCertificates loads and returns the tls.Certificate found at path if it
// exists and is valid. Otherwise, nil and an error is returned.
func loadCertificates(certPath, keyPath string) ([]tls.Certificate, error) {
cert, err := readFile(certPath)
if err != nil {
return nil, err
}
key, err := readFile(keyPath)
if err != nil {
return nil, err
}
crt, err := tls.X509KeyPair(cert, key)
if err != nil {
return nil, err
}
return []tls.Certificate{crt}, nil
}
// getenv returns a resolver that will apply an environment variable value
// associated with the first set key to a setting value. The conv function is
// used to convert between the environment variable value and the setting type.
//
// If the input setting to the resolver is set, the environment variable will
// not be applied.
//
// Any error returned from conv is sent to the OTel ErrorHandler and the
// setting will not be updated.
func getenv[T any](keys []string, conv func(string) (T, error)) resolver[T] {
return func(s setting[T]) setting[T] {
if s.Set {
// Passed, valid, options have precedence.
return s
}
for _, key := range keys {
if vStr := os.Getenv(key); vStr != "" {
v, err := conv(vStr)
if err == nil {
s.Value = v
s.Set = true
break
}
otel.Handle(fmt.Errorf("invalid %s value %s: %w", key, vStr, err))
}
}
return s
}
}
// convEndpoint converts s from a URL string to an endpoint if s is a valid
// URL. Otherwise, "" and an error are returned.
func convEndpoint(s string) (string, error) {
u, err := url.Parse(s)
if err != nil {
return "", err
}
return u.Host, nil
}
// convPathExact converts s from a URL string to the exact path if s is a valid
// URL. Otherwise, "" and an error are returned.
//
// If the path contained in s is empty, "/" is returned.
func convPathExact(s string) (string, error) {
u, err := url.Parse(s)
if err != nil {
return "", err
}
if u.Path == "" {
return "/", nil
}
return u.Path, nil
}
// convPath converts s from a URL string to an OTLP endpoint path if s is a
// valid URL. Otherwise, "" and an error are returned.
func convPath(s string) (string, error) {
u, err := url.Parse(s)
if err != nil {
return "", err
}
return u.Path + "/v1/logs", nil
}
// convInsecure parses s as a URL string and returns if the connection should
// use client transport security or not. If s is an invalid URL, false and an
// error are returned.
func convInsecure(s string) (bool, error) {
u, err := url.Parse(s)
if err != nil {
return false, err
}
return u.Scheme != "https", nil
}
// convHeaders converts the OTel environment variable header value s into a
// mapping of header key to value. If s is invalid a partial result and error
// are returned.
func convHeaders(s string) (map[string]string, error) {
out := make(map[string]string)
var err error
for _, header := range strings.Split(s, ",") {
rawKey, rawVal, found := strings.Cut(header, "=")
if !found {
err = errors.Join(err, fmt.Errorf("invalid header: %s", header))
continue
}
escKey, e := url.PathUnescape(rawKey)
if e != nil {
err = errors.Join(err, fmt.Errorf("invalid header key: %s", rawKey))
continue
}
key := strings.TrimSpace(escKey)
escVal, e := url.PathUnescape(rawVal)
if e != nil {
err = errors.Join(err, fmt.Errorf("invalid header value: %s", rawVal))
continue
}
val := strings.TrimSpace(escVal)
out[key] = val
}
return out, err
}
// convCompression returns the parsed compression encoded in s. NoCompression
// and an errors are returned if s is unknown.
func convCompression(s string) (Compression, error) {
switch s {
case "gzip":
return GzipCompression, nil
case "none", "":
return NoCompression, nil
}
return NoCompression, fmt.Errorf("unknown compression: %s", s)
}
// convDuration converts s into a duration of milliseconds. If s does not
// contain an integer, 0 and an error are returned.
func convDuration(s string) (time.Duration, error) {
d, err := strconv.Atoi(s)
if err != nil {
return 0, err
}
// OTel durations are defined in milliseconds.
return time.Duration(d) * time.Millisecond, nil
}
// fallback returns a resolve that will set a setting value to val if it is not
// already set.
//
// This is usually passed at the end of a resolver chain to ensure a default is
// applied if the setting has not already been set.
func fallback[T any](val T) resolver[T] {
return func(s setting[T]) setting[T] {
if !s.Set {
s.Value = val
s.Set = true
}
return s
}
}

View file

@ -0,0 +1,63 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
/*
Package otlploghttp provides an OTLP log exporter. The exporter uses HTTP to
transport OTLP protobuf payloads.
Exporter should be created using [New].
The environment variables described below can be used for configuration.
OTEL_EXPORTER_OTLP_ENDPOINT (default: "https://localhost:4318") -
target base URL ("/v1/logs" is appended) to which the exporter sends telemetry.
The value must contain a scheme ("http" or "https") and host.
The value may additionally contain a port and a path.
The value should not contain a query string or fragment.
The configuration can be overridden by OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
environment variable and by [WithEndpoint], [WithEndpointURL], [WithInsecure] options.
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT (default: "https://localhost:4318/v1/logs") -
target URL to which the exporter sends telemetry.
The value must contain a scheme ("http" or "https") and host.
The value may additionally contain a port and a path.
The value should not contain a query string or fragment.
The configuration can be overridden by [WithEndpoint], [WithEndpointURL], [WithInsecure], and [WithURLPath] options.
OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_LOGS_HEADERS (default: none) -
key-value pairs used as headers associated with HTTP requests.
The value is expected to be represented in a format matching the [W3C Baggage HTTP Header Content Format],
except that additional semi-colon delimited metadata is not supported.
Example value: "key1=value1,key2=value2".
OTEL_EXPORTER_OTLP_LOGS_HEADERS takes precedence over OTEL_EXPORTER_OTLP_HEADERS.
The configuration can be overridden by [WithHeaders] option.
OTEL_EXPORTER_OTLP_TIMEOUT, OTEL_EXPORTER_OTLP_LOGS_TIMEOUT (default: "10000") -
maximum time in milliseconds the OTLP exporter waits for each batch export.
OTEL_EXPORTER_OTLP_LOGS_TIMEOUT takes precedence over OTEL_EXPORTER_OTLP_TIMEOUT.
The configuration can be overridden by [WithTimeout] option.
OTEL_EXPORTER_OTLP_COMPRESSION, OTEL_EXPORTER_OTLP_LOGS_COMPRESSION (default: none) -
the compression strategy the exporter uses to compress the HTTP body.
Supported value: "gzip".
OTEL_EXPORTER_OTLP_LOGS_COMPRESSION takes precedence over OTEL_EXPORTER_OTLP_COMPRESSION.
The configuration can be overridden by [WithCompression] option.
OTEL_EXPORTER_OTLP_CERTIFICATE, OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE (default: none) -
the filepath to the trusted certificate to use when verifying a server's TLS credentials.
OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE takes precedence over OTEL_EXPORTER_OTLP_CERTIFICATE.
The configuration can be overridden by [WithTLSClientConfig] option.
OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE, OTEL_EXPORTER_OTLP_LOGS_CLIENT_CERTIFICATE (default: none) -
the filepath to the client certificate/chain trust for client's private key to use in mTLS communication in PEM format.
OTEL_EXPORTER_OTLP_LOGS_CLIENT_CERTIFICATE takes precedence over OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE.
The configuration can be overridden by [WithTLSClientConfig] option.
OTEL_EXPORTER_OTLP_CLIENT_KEY, OTEL_EXPORTER_OTLP_LOGS_CLIENT_KEY (default: none) -
the filepath to the client's private key to use in mTLS communication in PEM format.
OTEL_EXPORTER_OTLP_LOGS_CLIENT_KEY takes precedence over OTEL_EXPORTER_OTLP_CLIENT_KEY.
The configuration can be overridden by [WithTLSClientConfig] option.
[W3C Baggage HTTP Header Content Format]: https://www.w3.org/TR/baggage/#header-content
*/
package otlploghttp // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"

View file

@ -0,0 +1,73 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploghttp // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
import (
"context"
"sync/atomic"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp/internal/transform"
"go.opentelemetry.io/otel/sdk/log"
)
// Exporter is a OpenTelemetry log Exporter. It transports log data encoded as
// OTLP protobufs using HTTP.
// Exporter must be created with [New].
type Exporter struct {
client atomic.Pointer[client]
stopped atomic.Bool
}
// Compile-time check Exporter implements [log.Exporter].
var _ log.Exporter = (*Exporter)(nil)
// New returns a new [Exporter].
//
// It is recommended to use it with a [BatchProcessor]
// or other processor exporting records asynchronously.
func New(_ context.Context, options ...Option) (*Exporter, error) {
cfg := newConfig(options)
c, err := newHTTPClient(cfg)
if err != nil {
return nil, err
}
return newExporter(c, cfg)
}
func newExporter(c *client, _ config) (*Exporter, error) {
e := &Exporter{}
e.client.Store(c)
return e, nil
}
// Used for testing.
var transformResourceLogs = transform.ResourceLogs
// Export transforms and transmits log records to an OTLP receiver.
func (e *Exporter) Export(ctx context.Context, records []log.Record) error {
if e.stopped.Load() {
return nil
}
otlp := transformResourceLogs(records)
if otlp == nil {
return nil
}
return e.client.Load().UploadLogs(ctx, otlp)
}
// Shutdown shuts down the Exporter. Calls to Export or ForceFlush will perform
// no operation after this is called.
func (e *Exporter) Shutdown(ctx context.Context) error {
if e.stopped.Swap(true) {
return nil
}
e.client.Store(newNoopClient())
return nil
}
// ForceFlush does nothing. The Exporter holds no state.
func (e *Exporter) ForceFlush(ctx context.Context) error {
return nil
}

View file

@ -0,0 +1,145 @@
// Code created by gotmpl. DO NOT MODIFY.
// source: internal/shared/otlp/retry/retry.go.tmpl
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
// Package retry provides request retry functionality that can perform
// configurable exponential backoff for transient errors and honor any
// explicit throttle responses received.
package retry // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp/internal/retry"
import (
"context"
"fmt"
"time"
"github.com/cenkalti/backoff/v4"
)
// DefaultConfig are the recommended defaults to use.
var DefaultConfig = Config{
Enabled: true,
InitialInterval: 5 * time.Second,
MaxInterval: 30 * time.Second,
MaxElapsedTime: time.Minute,
}
// Config defines configuration for retrying batches in case of export failure
// using an exponential backoff.
type Config struct {
// Enabled indicates whether to not retry sending batches in case of
// export failure.
Enabled bool
// InitialInterval the time to wait after the first failure before
// retrying.
InitialInterval time.Duration
// MaxInterval is the upper bound on backoff interval. Once this value is
// reached the delay between consecutive retries will always be
// `MaxInterval`.
MaxInterval time.Duration
// MaxElapsedTime is the maximum amount of time (including retries) spent
// trying to send a request/batch. Once this value is reached, the data
// is discarded.
MaxElapsedTime time.Duration
}
// RequestFunc wraps a request with retry logic.
type RequestFunc func(context.Context, func(context.Context) error) error
// EvaluateFunc returns if an error is retry-able and if an explicit throttle
// duration should be honored that was included in the error.
//
// The function must return true if the error argument is retry-able,
// otherwise it must return false for the first return parameter.
//
// The function must return a non-zero time.Duration if the error contains
// explicit throttle duration that should be honored, otherwise it must return
// a zero valued time.Duration.
type EvaluateFunc func(error) (bool, time.Duration)
// RequestFunc returns a RequestFunc using the evaluate function to determine
// if requests can be retried and based on the exponential backoff
// configuration of c.
func (c Config) RequestFunc(evaluate EvaluateFunc) RequestFunc {
if !c.Enabled {
return func(ctx context.Context, fn func(context.Context) error) error {
return fn(ctx)
}
}
return func(ctx context.Context, fn func(context.Context) error) error {
// Do not use NewExponentialBackOff since it calls Reset and the code here
// must call Reset after changing the InitialInterval (this saves an
// unnecessary call to Now).
b := &backoff.ExponentialBackOff{
InitialInterval: c.InitialInterval,
RandomizationFactor: backoff.DefaultRandomizationFactor,
Multiplier: backoff.DefaultMultiplier,
MaxInterval: c.MaxInterval,
MaxElapsedTime: c.MaxElapsedTime,
Stop: backoff.Stop,
Clock: backoff.SystemClock,
}
b.Reset()
for {
err := fn(ctx)
if err == nil {
return nil
}
retryable, throttle := evaluate(err)
if !retryable {
return err
}
bOff := b.NextBackOff()
if bOff == backoff.Stop {
return fmt.Errorf("max retry time elapsed: %w", err)
}
// Wait for the greater of the backoff or throttle delay.
var delay time.Duration
if bOff > throttle {
delay = bOff
} else {
elapsed := b.GetElapsedTime()
if b.MaxElapsedTime != 0 && elapsed+throttle > b.MaxElapsedTime {
return fmt.Errorf("max retry time would elapse: %w", err)
}
delay = throttle
}
if ctxErr := waitFunc(ctx, delay); ctxErr != nil {
return fmt.Errorf("%w: %w", ctxErr, err)
}
}
}
}
// Allow override for testing.
var waitFunc = wait
// wait takes the caller's context, and the amount of time to wait. It will
// return nil if the timer fires before or at the same time as the context's
// deadline. This indicates that the call can be retried.
func wait(ctx context.Context, delay time.Duration) error {
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
// Handle the case where the timer and context deadline end
// simultaneously by prioritizing the timer expiration nil value
// response.
select {
case <-timer.C:
default:
return ctx.Err()
}
case <-timer.C:
}
return nil
}

View file

@ -0,0 +1,391 @@
// Code created by gotmpl. DO NOT MODIFY.
// source: internal/shared/otlp/otlplog/transform/log.go.tmpl
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
// Package transform provides transformation functionality from the
// sdk/log data-types into OTLP data-types.
package transform // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp/internal/transform"
import (
"time"
cpb "go.opentelemetry.io/proto/otlp/common/v1"
lpb "go.opentelemetry.io/proto/otlp/logs/v1"
rpb "go.opentelemetry.io/proto/otlp/resource/v1"
"go.opentelemetry.io/otel/attribute"
api "go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/sdk/instrumentation"
"go.opentelemetry.io/otel/sdk/log"
)
// ResourceLogs returns an slice of OTLP ResourceLogs generated from records.
func ResourceLogs(records []log.Record) []*lpb.ResourceLogs {
if len(records) == 0 {
return nil
}
resMap := make(map[attribute.Distinct]*lpb.ResourceLogs)
type key struct {
r attribute.Distinct
is instrumentation.Scope
}
scopeMap := make(map[key]*lpb.ScopeLogs)
var resources int
for _, r := range records {
res := r.Resource()
rKey := res.Equivalent()
scope := r.InstrumentationScope()
k := key{
r: rKey,
is: scope,
}
sl, iOk := scopeMap[k]
if !iOk {
sl = new(lpb.ScopeLogs)
var emptyScope instrumentation.Scope
if scope != emptyScope {
sl.Scope = &cpb.InstrumentationScope{
Name: scope.Name,
Version: scope.Version,
Attributes: AttrIter(scope.Attributes.Iter()),
}
sl.SchemaUrl = scope.SchemaURL
}
scopeMap[k] = sl
}
sl.LogRecords = append(sl.LogRecords, LogRecord(r))
rl, rOk := resMap[rKey]
if !rOk {
resources++
rl = new(lpb.ResourceLogs)
if res.Len() > 0 {
rl.Resource = &rpb.Resource{
Attributes: AttrIter(res.Iter()),
}
}
rl.SchemaUrl = res.SchemaURL()
resMap[rKey] = rl
}
if !iOk {
rl.ScopeLogs = append(rl.ScopeLogs, sl)
}
}
// Transform the categorized map into a slice
resLogs := make([]*lpb.ResourceLogs, 0, resources)
for _, rl := range resMap {
resLogs = append(resLogs, rl)
}
return resLogs
}
// LogRecord returns an OTLP LogRecord generated from record.
func LogRecord(record log.Record) *lpb.LogRecord {
r := &lpb.LogRecord{
TimeUnixNano: timeUnixNano(record.Timestamp()),
ObservedTimeUnixNano: timeUnixNano(record.ObservedTimestamp()),
EventName: record.EventName(),
SeverityNumber: SeverityNumber(record.Severity()),
SeverityText: record.SeverityText(),
Body: LogAttrValue(record.Body()),
Attributes: make([]*cpb.KeyValue, 0, record.AttributesLen()),
Flags: uint32(record.TraceFlags()),
// TODO: DroppedAttributesCount: /* ... */,
}
record.WalkAttributes(func(kv api.KeyValue) bool {
r.Attributes = append(r.Attributes, LogAttr(kv))
return true
})
if tID := record.TraceID(); tID.IsValid() {
r.TraceId = tID[:]
}
if sID := record.SpanID(); sID.IsValid() {
r.SpanId = sID[:]
}
return r
}
// timeUnixNano returns t as a Unix time, the number of nanoseconds elapsed
// since January 1, 1970 UTC as uint64. The result is undefined if the Unix
// time in nanoseconds cannot be represented by an int64 (a date before the
// year 1678 or after 2262). timeUnixNano on the zero Time returns 0. The
// result does not depend on the location associated with t.
func timeUnixNano(t time.Time) uint64 {
nano := t.UnixNano()
if nano < 0 {
return 0
}
return uint64(nano) // nolint:gosec // Overflow checked.
}
// AttrIter transforms an [attribute.Iterator] into OTLP key-values.
func AttrIter(iter attribute.Iterator) []*cpb.KeyValue {
l := iter.Len()
if l == 0 {
return nil
}
out := make([]*cpb.KeyValue, 0, l)
for iter.Next() {
out = append(out, Attr(iter.Attribute()))
}
return out
}
// Attrs transforms a slice of [attribute.KeyValue] into OTLP key-values.
func Attrs(attrs []attribute.KeyValue) []*cpb.KeyValue {
if len(attrs) == 0 {
return nil
}
out := make([]*cpb.KeyValue, 0, len(attrs))
for _, kv := range attrs {
out = append(out, Attr(kv))
}
return out
}
// Attr transforms an [attribute.KeyValue] into an OTLP key-value.
func Attr(kv attribute.KeyValue) *cpb.KeyValue {
return &cpb.KeyValue{Key: string(kv.Key), Value: AttrValue(kv.Value)}
}
// AttrValue transforms an [attribute.Value] into an OTLP AnyValue.
func AttrValue(v attribute.Value) *cpb.AnyValue {
av := new(cpb.AnyValue)
switch v.Type() {
case attribute.BOOL:
av.Value = &cpb.AnyValue_BoolValue{
BoolValue: v.AsBool(),
}
case attribute.BOOLSLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: boolSliceValues(v.AsBoolSlice()),
},
}
case attribute.INT64:
av.Value = &cpb.AnyValue_IntValue{
IntValue: v.AsInt64(),
}
case attribute.INT64SLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: int64SliceValues(v.AsInt64Slice()),
},
}
case attribute.FLOAT64:
av.Value = &cpb.AnyValue_DoubleValue{
DoubleValue: v.AsFloat64(),
}
case attribute.FLOAT64SLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: float64SliceValues(v.AsFloat64Slice()),
},
}
case attribute.STRING:
av.Value = &cpb.AnyValue_StringValue{
StringValue: v.AsString(),
}
case attribute.STRINGSLICE:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: stringSliceValues(v.AsStringSlice()),
},
}
default:
av.Value = &cpb.AnyValue_StringValue{
StringValue: "INVALID",
}
}
return av
}
func boolSliceValues(vals []bool) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_BoolValue{
BoolValue: v,
},
}
}
return converted
}
func int64SliceValues(vals []int64) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_IntValue{
IntValue: v,
},
}
}
return converted
}
func float64SliceValues(vals []float64) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_DoubleValue{
DoubleValue: v,
},
}
}
return converted
}
func stringSliceValues(vals []string) []*cpb.AnyValue {
converted := make([]*cpb.AnyValue, len(vals))
for i, v := range vals {
converted[i] = &cpb.AnyValue{
Value: &cpb.AnyValue_StringValue{
StringValue: v,
},
}
}
return converted
}
// Attrs transforms a slice of [api.KeyValue] into OTLP key-values.
func LogAttrs(attrs []api.KeyValue) []*cpb.KeyValue {
if len(attrs) == 0 {
return nil
}
out := make([]*cpb.KeyValue, 0, len(attrs))
for _, kv := range attrs {
out = append(out, LogAttr(kv))
}
return out
}
// LogAttr transforms an [api.KeyValue] into an OTLP key-value.
func LogAttr(attr api.KeyValue) *cpb.KeyValue {
return &cpb.KeyValue{
Key: attr.Key,
Value: LogAttrValue(attr.Value),
}
}
// LogAttrValues transforms a slice of [api.Value] into an OTLP []AnyValue.
func LogAttrValues(vals []api.Value) []*cpb.AnyValue {
if len(vals) == 0 {
return nil
}
out := make([]*cpb.AnyValue, 0, len(vals))
for _, v := range vals {
out = append(out, LogAttrValue(v))
}
return out
}
// LogAttrValue transforms an [api.Value] into an OTLP AnyValue.
func LogAttrValue(v api.Value) *cpb.AnyValue {
av := new(cpb.AnyValue)
switch v.Kind() {
case api.KindBool:
av.Value = &cpb.AnyValue_BoolValue{
BoolValue: v.AsBool(),
}
case api.KindInt64:
av.Value = &cpb.AnyValue_IntValue{
IntValue: v.AsInt64(),
}
case api.KindFloat64:
av.Value = &cpb.AnyValue_DoubleValue{
DoubleValue: v.AsFloat64(),
}
case api.KindString:
av.Value = &cpb.AnyValue_StringValue{
StringValue: v.AsString(),
}
case api.KindBytes:
av.Value = &cpb.AnyValue_BytesValue{
BytesValue: v.AsBytes(),
}
case api.KindSlice:
av.Value = &cpb.AnyValue_ArrayValue{
ArrayValue: &cpb.ArrayValue{
Values: LogAttrValues(v.AsSlice()),
},
}
case api.KindMap:
av.Value = &cpb.AnyValue_KvlistValue{
KvlistValue: &cpb.KeyValueList{
Values: LogAttrs(v.AsMap()),
},
}
default:
av.Value = &cpb.AnyValue_StringValue{
StringValue: "INVALID",
}
}
return av
}
// SeverityNumber transforms a [log.Severity] into an OTLP SeverityNumber.
func SeverityNumber(s api.Severity) lpb.SeverityNumber {
switch s {
case api.SeverityTrace:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE
case api.SeverityTrace2:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE2
case api.SeverityTrace3:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE3
case api.SeverityTrace4:
return lpb.SeverityNumber_SEVERITY_NUMBER_TRACE4
case api.SeverityDebug:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG
case api.SeverityDebug2:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG2
case api.SeverityDebug3:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG3
case api.SeverityDebug4:
return lpb.SeverityNumber_SEVERITY_NUMBER_DEBUG4
case api.SeverityInfo:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO
case api.SeverityInfo2:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO2
case api.SeverityInfo3:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO3
case api.SeverityInfo4:
return lpb.SeverityNumber_SEVERITY_NUMBER_INFO4
case api.SeverityWarn:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN
case api.SeverityWarn2:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN2
case api.SeverityWarn3:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN3
case api.SeverityWarn4:
return lpb.SeverityNumber_SEVERITY_NUMBER_WARN4
case api.SeverityError:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR
case api.SeverityError2:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR2
case api.SeverityError3:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR3
case api.SeverityError4:
return lpb.SeverityNumber_SEVERITY_NUMBER_ERROR4
case api.SeverityFatal:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL
case api.SeverityFatal2:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL2
case api.SeverityFatal3:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL3
case api.SeverityFatal4:
return lpb.SeverityNumber_SEVERITY_NUMBER_FATAL4
}
return lpb.SeverityNumber_SEVERITY_NUMBER_UNSPECIFIED
}

View file

@ -0,0 +1,9 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0
package otlploghttp // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
// Version is the current release version of the OpenTelemetry OTLP over HTTP/protobuf logs exporter in use.
func Version() string {
return "0.11.0"
}