[feature] simpler cache size configuration (#2051)

* add automatic cache max size generation based on ratios of a singular fixed memory target Signed-off-by: kim <grufwub@gmail.com> * remove now-unused cache max-size config variables Signed-off-by: kim <grufwub@gmail.com> * slight ratio tweak Signed-off-by: kim <grufwub@gmail.com> * remove unused visibility config var Signed-off-by: kim <grufwub@gmail.com> * add secret little ratio config trick Signed-off-by: kim <grufwub@gmail.com> * fixed a word Signed-off-by: kim <grufwub@gmail.com> * update cache library to remove use of TTL in result caches + slice cache Signed-off-by: kim <grufwub@gmail.com> * update other cache usages to use correct interface Signed-off-by: kim <grufwub@gmail.com> * update example config to explain the cache memory target Signed-off-by: kim <grufwub@gmail.com> * update env parsing test with new config values Signed-off-by: kim <grufwub@gmail.com> * do some ratio twiddling Signed-off-by: kim <grufwub@gmail.com> * add missing header * update envparsing with latest defaults Signed-off-by: kim <grufwub@gmail.com> * update size calculations to take into account result cache, simple cache and extra map overheads Signed-off-by: kim <grufwub@gmail.com> * tweak the ratios some more Signed-off-by: kim <grufwub@gmail.com> * more nan rampaging Signed-off-by: kim <grufwub@gmail.com> * fix envparsing script Signed-off-by: kim <grufwub@gmail.com> * update cache library, add sweep function to keep caches trim Signed-off-by: kim <grufwub@gmail.com> * sweep caches once a minute Signed-off-by: kim <grufwub@gmail.com> * add a regular job to sweep caches and keep under 80% utilisation Signed-off-by: kim <grufwub@gmail.com> * remove dead code Signed-off-by: kim <grufwub@gmail.com> * add new size library used to libraries section of readme Signed-off-by: kim <grufwub@gmail.com> * add better explanations for the mem-ratio numbers Signed-off-by: kim <grufwub@gmail.com> * update go-cache Signed-off-by: kim <grufwub@gmail.com> * library version bump Signed-off-by: kim <grufwub@gmail.com> * update cache.result{} size model estimation Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2025-11-02 02:52:25 -06:00 · 2023-08-03 10:34:35 +01:00 · 2023-08-03 10:34:35 +01:00 · 00adf18c24
commit 00adf18c24
parent e8a20f587c
32 changed files with 2115 additions and 2330 deletions
--- a/vendor/github.com/DmitriyVTitov/size/README.md
+++ b/vendor/github.com/DmitriyVTitov/size/README.md
@ -0,0 +1,48 @@
+# size - calculates variable's memory consumption at runtime
+
+### Part of the [Transflow Project](http://transflow.ru/)
+
+Sometimes you may need a tool to measure the size of object in your Go program at runtime. This package makes an attempt to do so. Package based on `binary.Size()` from Go standard library.
+
+Features:
+- supports non-fixed size variables and struct fields: `struct`, `int`, `slice`, `string`, `map`;
+- supports complex types including structs with non-fixed size fields;
+- supports all basic types (numbers, bool);
+- supports `chan` and `interface`;
+- supports pointers;
+- implements infinite recursion detection (i.e. pointer inside struct field references to parent struct).
+
+### Usage example
+
+```
+package main
+
+import (
+	"fmt"
+
+	// Use latest tag.
+	"github.com/DmitriyVTitov/size"
+)
+
+func main() {
+	a := struct {
+		a int
+		b string
+		c bool
+		d int32
+		e []byte
+		f [3]int64
+	}{
+		a: 10,                    // 8 bytes
+		b: "Text",                // 16 (string itself) + 4 = 20 bytes
+		c: true,                  // 1 byte
+		d: 25,                    // 4 bytes
+		e: []byte{'c', 'd', 'e'}, // 24 (slice itself) + 3 = 27 bytes
+		f: [3]int64{1, 2, 3},     // 3 * 8 = 24 bytes
+	} // 84 + 3 (padding) = 87 bytes
+
+	fmt.Println(size.Of(a))
+}
+
+// Output: 87
+```