mirror of
https://github.com/superseriousbusiness/gotosocial.git
synced 2025-10-28 16:02:25 -05:00
[chore/docs] Fix Prometheus metric names for Gin, include example Grafana dash, update docs (#4443)
# Description > If this is a code change, please include a summary of what you've coded, and link to the issue(s) it closes/implements. > > If this is a documentation change, please briefly describe what you've changed and why. This pull request updates some of our inconsistent metric naming, and adds an example Grafana dashboard using all the most up-to-date metrics names, and updates our docs to describe the latest way of setting up metrics. Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/4362 Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/4055 ## Checklist Please put an x inside each checkbox to indicate that you've read and followed it: `[ ]` -> `[x]` If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want). - [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md). - [x] I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat. - [x] I/we have not leveraged AI to create the proposed changes. - [x] I/we have performed a self-review of added code. - [x] I/we have written code that is legible and maintainable by others. - [x] I/we have commented the added code, particularly in hard-to-understand areas. - [x] I/we have made any necessary changes to documentation. - [ ] I/we have added tests that cover new code. - [x] I/we have run tests and they pass locally with the changes. - [x] I/we have run `go fmt ./...` and `golangci-lint run`. Co-authored-by: kim <grufwub@gmail.com> Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4443 Reviewed-by: kim <gruf@noreply.codeberg.org> Co-authored-by: tobi <tobi.smethurst@protonmail.com> Co-committed-by: tobi <tobi.smethurst@protonmail.com>
This commit is contained in:
tobi
tobi
parent
6607e1c944
commit
82216281ce
32 changed files with 2315 additions and 4760 deletions
|
|
@ -1,32 +1,89 @@
|
|||
# Metrics
|
||||
|
||||
GoToSocial comes with [OpenTelemetry][otel] based metrics. The metrics are exposed using the [Prometheus exposition format][prom] on the `/metrics` path. The configuration settings are documented in the [Observability configuration reference][obs].
|
||||
GoToSocial uses the [OpenTelemetry][otel] Go SDK to enable instance admins to expose runtime metrics in the Prometheus metrics format.
|
||||
|
||||
Currently the following metrics are collected:
|
||||
Currently, the following metrics are collected:
|
||||
|
||||
* Go performance and runtime metrics
|
||||
* Gin (HTTP) metrics
|
||||
* Gin (HTTP server) metrics
|
||||
* Bun (database) metrics
|
||||
|
||||
Metrics can be enable with the following configuration:
|
||||
## Enabling metrics
|
||||
|
||||
To enable metrics, first set the `metrics-enabled` configuration value to `true` in your config.yaml file:
|
||||
|
||||
```yaml
|
||||
metrics-enabled: true
|
||||
```
|
||||
|
||||
Though metrics do not contain anything privacy sensitive, you may not want to allow just anyone to view and scrape operational metrics of your instance.
|
||||
Then, you will need to set some additional environment variables on the GoToSocial process in order to configure OpenTelemetry to expose metrics in the Prometheus format:
|
||||
|
||||
```env
|
||||
OTEL_METRICS_PRODUCERS=prometheus
|
||||
OTEL_METRICS_EXPORTER=prometheus
|
||||
```
|
||||
|
||||
By default, this configuration will instantiate an additional HTTP server running alongside the standard GoToSocial HTTP server, which exposes a Prometheus metrics endpoint at `localhost:9464/metrics`.
|
||||
|
||||
!!! tip
|
||||
If you are running GoToSocial using the [example systemd service definition](../../example/gotosocial.service), you can easily set these environment variables by uncommenting the relevant two lines in that file, and reloading + restarting the service.
|
||||
|
||||
If you wish, you can further customize this metrics HTTP server by using the following environment variables to change the host and port:
|
||||
|
||||
```env
|
||||
OTEL_EXPORTER_PROMETHEUS_HOST=example.org
|
||||
OTEL_EXPORTER_PROMETHEUS_PORT=9999
|
||||
```
|
||||
|
||||
## Serving metrics to the outside world
|
||||
|
||||
If you have deployed GoToSocial in a "bare-metal" fashion without a reverse proxy, you can expose the metrics endpoint to the outside world by setting `OTEL_EXPORTER_PROMETHEUS_HOST` to your host value. For example, if your GtS instance `host` configuration value is set to `example.org`, you should set `OTEL_EXPORTER_PROMETHEUS_HOST=example.org`. You should then be able to access your metrics at `http://example.org:9464/metrics`. GoToSocial running in this fashion will not serve LetsEncrypt certificates at the metrics endpoint, so you will be limited to using HTTP rather than HTTPS.
|
||||
|
||||
If you are using a reverse proxy like Nginx, you can expose the metrics endpoint to the outside world with HTTPS certificates, by putting an additional location stanza in your Nginx configuration above the catch-all `location /` stanza:
|
||||
|
||||
```nginx
|
||||
location /metrics {
|
||||
proxy_pass http://127.0.0.1:9464;
|
||||
}
|
||||
```
|
||||
|
||||
This will instruct Nginx to forward requests to `example.org/metrics` to the separate Prometheus server running on port 9464.
|
||||
|
||||
## Enabling basic authentication
|
||||
|
||||
You can enable basic authentication for the metrics endpoint. On the GoToSocial, side you'll need the following configuration:
|
||||
Although there is no sensitive data contained in the OTEL runtime statistics exported by Prometheus, you may nevertheless wish to gate access to the `/metrics` endpoint behind some kind of authentication, to prevent every Tom, Dick, and Harry from looking at your runtime stats.
|
||||
|
||||
```yaml
|
||||
metrics-auth-enabled: true
|
||||
metrics-auth-username: some_username
|
||||
metrics-auth-password: some_password
|
||||
You can do this by configuring your reverse proxy to require basic authentication for access to `/metrics`.
|
||||
|
||||
In Nginx, for example, you could do this by creating an `htpasswd` file alongside your site in the `sites-available` directory of Nginx, and instructing Nginx to use that file to gate access.
|
||||
|
||||
Assuming you followed the [guide for setting up Nginx as your reverse proxy](../getting_started/reverse_proxy/nginx.md), you will already have a file for your Nginx service definition at `/etc/nginx/sites-available/example.org`, where `example.org` is the hostname of your instance.
|
||||
|
||||
You can create an `htpasswd` file alongside this file using the following command:
|
||||
|
||||
```bash
|
||||
htpasswd -c /etc/nginx/sites-available/example.org.htpasswd username
|
||||
```
|
||||
|
||||
You can scrape that endpoint with a Prometheus instance using the following configuration in your `scrape_configs`:
|
||||
In the command, be sure to replace `example.org` with your hostname, and `username` with whatever username you want to use.
|
||||
|
||||
Now, edit `/etc/nginx/sites-available/example.org` and update your `/metrics` stanza to use the `httpasswd` file. After editing it should look something like this:
|
||||
|
||||
```nginx
|
||||
location /metrics {
|
||||
proxy_pass http://127.0.0.1:9464;
|
||||
auth_basic "Metrics";
|
||||
auth_basic_user_file /etc/nginx/sites-available/example.org.htpasswd;
|
||||
}
|
||||
```
|
||||
|
||||
Again, replace `example.org` in that snippet with your instance hostname.
|
||||
|
||||
When you're finished editing, reload + restart Nginx, and you should see a basic authentication prompt when visiting the `/metrics` endpoint of your instance in your browser.
|
||||
|
||||
## Prometheus scrape configuration
|
||||
|
||||
You can scrape your `/metrics` endpoint with a Prometheus instance using the following configuration in your `scrape_configs`:
|
||||
|
||||
```yaml
|
||||
- job_name: gotosocial
|
||||
|
|
@ -40,18 +97,12 @@ You can scrape that endpoint with a Prometheus instance using the following conf
|
|||
- example.org
|
||||
```
|
||||
|
||||
## Blocking external scraping
|
||||
Change `example.org` to your hostname in the above snippet. If you are not using HTTPS, change the `scheme` value to `http`. If you are not using basic authentication, you can remove the `basic_auth` section. If you are not using a reverse proxy, and metrics are exposed on port 9464, add the port to the host (eg., `example.org` -> `example.org:9464`).
|
||||
|
||||
When running with a reverse proxy you can use it to block external access to metrics. You can use this approach if your Prometheus scraper runs on the same machine as your GoToSocial instance and can thus access it internally.
|
||||
## Viewing metrics on Grafana
|
||||
|
||||
For example with nginx, block the `/metrics` endpoint by returning a 404:
|
||||
|
||||
```nginx
|
||||
location /metrics {
|
||||
return 404;
|
||||
}
|
||||
```
|
||||
Instructions on how to set up Grafana are beyond the scope of this document. However, once you have set up a Grafana to pull from your Prometheus instance, you can import the [example Grafana dashboard](https://codeberg.org/superseriousbusiness/gotosocial/raw/branch/main/example/metrics/gotosocial_grafana_dashboard.json) into your Grafana frontend to easily view GoToSocial Go runtime and HTTP metrics.
|
||||
|
||||
[otel]: https://opentelemetry.io/
|
||||
[prom]: https://prometheus.io/docs/instrumenting/exposition_formats/
|
||||
[obs]: ../configuration/observability.md
|
||||
[obs]: ../configuration/observability_and_metrics.md
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ tracing-insecure-transport: true
|
|||
```
|
||||
|
||||
[otel]: https://opentelemetry.io/
|
||||
[obs]: ../configuration/observability.md
|
||||
[obs]: ../configuration/observability_and_metrics.md
|
||||
[tempo]: https://grafana.com/oss/tempo/
|
||||
[grafana]: https://grafana.com/oss/grafana/
|
||||
[ext]: https://codeberg.org/superseriousbusiness/gotosocial/tree/main/example/tracing
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue